[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:22 dist-test-slave-trms krb5kdc[25222](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:22 dist-test-slave-trms krb5kdc[25222](info): set up 2 sockets
Sep 25 11:00:22 dist-test-slave-trms krb5kdc[25222](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260925 11:00:23.778574 25215 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.008s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:00:23 dist-test-slave-trms krb5kdc[25222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:00:23.932235 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43781
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:43781
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:24.235392 25235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:24.235683 25235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:24.235777 25235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:24.242672 25235 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:24.242766 25235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:24.242815 25235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:24.242874 25235 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:24.242916 25235 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:24.252554 25235 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:43781
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:24.253800 25235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:24.255174 25235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:24.266803 25240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:24.266742 25243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:24.266773 25241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:24.267639 25235 server_base.cc:1034] running on GCE node
I20260925 11:00:24.268376 25235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:24.269800 25235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:24.271003 25235 hybrid_clock.cc:648] HybridClock initialized: now 1790334024270950 us; error 45 us; skew 500 ppm
I20260925 11:00:24.271344 25235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:24 dist-test-slave-trms krb5kdc[25222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334024, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:24.274283 25235 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:00:24.275755 25235 webserver.cc:460] Webserver started at http://127.24.159.254:42943/ using document root <none> and password file <none>
I20260925 11:00:24.276429 25235 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:24.276547 25235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:24.276827 25235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:24.279835 25235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "8c41e457a61c4cc6ae1c40b58b114b8c"
format_stamp: "Formatted at 2026-09-25 11:00:24 on dist-test-slave-trms"
server_key: "f320afc7c58dd97d328288a1fc4f829d"
server_key_iv: "602c48af1aa3a69cec7ebf5ab3fbfb0c"
server_key_version: "encryptionkey@0"
I20260925 11:00:24.280575 25235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "8c41e457a61c4cc6ae1c40b58b114b8c"
format_stamp: "Formatted at 2026-09-25 11:00:24 on dist-test-slave-trms"
server_key: "f320afc7c58dd97d328288a1fc4f829d"
server_key_iv: "602c48af1aa3a69cec7ebf5ab3fbfb0c"
server_key_version: "encryptionkey@0"
I20260925 11:00:24.285681 25235 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 11:00:24.289112 25250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:24.290475 25235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:24.290642 25235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "8c41e457a61c4cc6ae1c40b58b114b8c"
format_stamp: "Formatted at 2026-09-25 11:00:24 on dist-test-slave-trms"
server_key: "f320afc7c58dd97d328288a1fc4f829d"
server_key_iv: "602c48af1aa3a69cec7ebf5ab3fbfb0c"
server_key_version: "encryptionkey@0"
I20260925 11:00:24.290817 25235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:24.332605 25235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:24.336853 25235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:24.337122 25235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:24.353929 25235 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:43781
I20260925 11:00:24.353950 25301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:43781 every 8 connection(s)
I20260925 11:00:24.355307 25235 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:24.357128 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 25235
I20260925 11:00:24.357451 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:00:24.359869 25215 external_mini_cluster.cc:1428] Setting key d90a85edefa7f35718a8a28bd665a8b7
I20260925 11:00:24.359967 25302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:24.371165 25302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c: Bootstrap starting.
I20260925 11:00:24.374423 25302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:24.375797 25302 log.cc:824] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:24.378633 25302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c: No bootstrap required, opened a new log
I20260925 11:00:24.386967 25302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c41e457a61c4cc6ae1c40b58b114b8c" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43781 } }
I20260925 11:00:24.387306 25302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:24.387398 25302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c41e457a61c4cc6ae1c40b58b114b8c, State: Initialized, Role: FOLLOWER
I20260925 11:00:24.387943 25302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c41e457a61c4cc6ae1c40b58b114b8c" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43781 } }
I20260925 11:00:24.388146 25302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:24.388278 25302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:24.388435 25302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [term 0 FOLLOWER]: Advancing to term 1
Sep 25 11:00:24 dist-test-slave-trms krb5kdc[25222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:24.391347 25302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c41e457a61c4cc6ae1c40b58b114b8c" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43781 } }
I20260925 11:00:24.391970 25302 leader_election.cc:302] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c41e457a61c4cc6ae1c40b58b114b8c; no voters:
I20260925 11:00:24.392937 25302 leader_election.cc:288] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:24.393072 25307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:24.394447 25307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [term 1 LEADER]: Becoming Leader. State: Replica: 8c41e457a61c4cc6ae1c40b58b114b8c, State: Running, Role: LEADER
I20260925 11:00:24.395020 25307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c41e457a61c4cc6ae1c40b58b114b8c" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43781 } }
I20260925 11:00:24.395454 25302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:24.401104 25304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:24.364463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.24.159.254:43781)
0925 11:00:24.365373 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:24.365389 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:24.366687 (+ 1298us) server_negotiation.cc:407] Connection header received
0925 11:00:24.375798 (+ 9111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:24.375826 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:24.376349 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:24.376735 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:24.379628 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:24.381669 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:24.383286 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:24.384352 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:24.384534 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:24.390055 (+ 5521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:24.390066 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:24.390085 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:24.390134 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:24.394579 (+ 4445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:24.395658 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:24.395667 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:24.395674 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:24.395888 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:24.396442 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:24.396448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:24.396452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:24.397244 (+ 792us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:24.397561 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:24.398631 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0925 11:00:24.399097 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":642,"thread_start_us":393,"threads_started":1}
I20260925 11:00:24.403752 25308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c41e457a61c4cc6ae1c40b58b114b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c41e457a61c4cc6ae1c40b58b114b8c" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43781 } } }
I20260925 11:00:24.404153 25308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:24.403724 25309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c41e457a61c4cc6ae1c40b58b114b8c. Latest consensus state: current_term: 1 leader_uuid: "8c41e457a61c4cc6ae1c40b58b114b8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c41e457a61c4cc6ae1c40b58b114b8c" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43781 } } }
I20260925 11:00:24.404670 25309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c [sys.catalog]: This master's current role is: LEADER
W20260925 11:00:24.413177 25324 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:00:24.413314 25324 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:00:24.413419 25316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:24.415489 25316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:24.424027 25316 catalog_manager.cc:1347] Generated new cluster ID: f03df9679cfe4472863300f9d254ee95
I20260925 11:00:24.424139 25316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:24.439045 25316 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:24.440320 25316 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:24.450464 25316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c41e457a61c4cc6ae1c40b58b114b8c: Generated new TSK 0
I20260925 11:00:24.451299 25316 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:00:24.496156 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43781
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:24.833921 25329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:24.834184 25329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:24.834321 25329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:24.841399 25329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:24.841501 25329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:24.841624 25329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:00:24.851217 25329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43781
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:24.852473 25329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:24.853718 25329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:24.865615 25335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:24.865649 25334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:24.865676 25337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:24.866379 25329 server_base.cc:1034] running on GCE node
I20260925 11:00:24.866958 25329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:24.867908 25329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:24.869087 25329 hybrid_clock.cc:648] HybridClock initialized: now 1790334024869029 us; error 55 us; skew 500 ppm
I20260925 11:00:24.869369 25329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:24 dist-test-slave-trms krb5kdc[25222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334024, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:24.872253 25329 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:00:24.873627 25329 webserver.cc:460] Webserver started at http://127.24.159.193:35281/ using document root <none> and password file <none>
I20260925 11:00:24.874343 25329 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:24.874464 25329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:24.874747 25329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:24.877557 25329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "12a94841b4694a2497d4f952d17a2a19"
format_stamp: "Formatted at 2026-09-25 11:00:24 on dist-test-slave-trms"
server_key: "23823f7085eb452223d5445e0a3e451e"
server_key_iv: "9ba1ad5d5abf10d315dd50e064ba0930"
server_key_version: "encryptionkey@0"
I20260925 11:00:24.878337 25329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "12a94841b4694a2497d4f952d17a2a19"
format_stamp: "Formatted at 2026-09-25 11:00:24 on dist-test-slave-trms"
server_key: "23823f7085eb452223d5445e0a3e451e"
server_key_iv: "9ba1ad5d5abf10d315dd50e064ba0930"
server_key_version: "encryptionkey@0"
I20260925 11:00:24.883311 25329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:00:24.886601 25344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:24.887895 25329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:00:24.888036 25329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "12a94841b4694a2497d4f952d17a2a19"
format_stamp: "Formatted at 2026-09-25 11:00:24 on dist-test-slave-trms"
server_key: "23823f7085eb452223d5445e0a3e451e"
server_key_iv: "9ba1ad5d5abf10d315dd50e064ba0930"
server_key_version: "encryptionkey@0"
I20260925 11:00:24.888213 25329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:24.911163 25329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:24.916154 25329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:24.916400 25329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:24.917456 25329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:24.919183 25329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:24.919276 25329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:24.919395 25329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:24.919458 25329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:24.952114 25329 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:42969
I20260925 11:00:24.952215 25456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:42969 every 8 connection(s)
I20260925 11:00:24.953465 25329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:00:24.953971 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 25329
I20260925 11:00:24.954229 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:00:24.954967 25215 external_mini_cluster.cc:1428] Setting key 09a8155aafc16f0809ff6e7420146f34
Sep 25 11:00:24 dist-test-slave-trms krb5kdc[25222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334024, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
I20260925 11:00:24.974591 25459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:24.956256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:55203 (local address 127.24.159.254:43781)
0925 11:00:24.956714 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:24.956721 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:24.957867 (+ 1146us) server_negotiation.cc:407] Connection header received
0925 11:00:24.960355 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:24.960361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:24.960448 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:24.960547 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:24.962053 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:24.963104 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:24.964649 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:24.965241 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:24.965348 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:24.968919 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:24.968927 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:24.968932 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:24.968974 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:24.971995 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:24.972649 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:24.972652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:24.972655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:24.972766 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:24.973081 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:24.973083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:24.973085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:24.973328 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:24.973438 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:24.974076 (+ 638us) server_negotiation.cc:299] Negotiation successful
0925 11:00:24.974322 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":825,"threads_started":1}
I20260925 11:00:24.975803 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:24.956837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43781 (local address 127.24.159.193:55203)
0925 11:00:24.957707 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:24.957760 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:24.960076 (+ 2316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:24.960788 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:24.960796 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:24.961310 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:24.961884 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:24.961903 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:24.963257 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:24.963261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:24.964497 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:24.964509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:24.965444 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:24.965449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:24.965648 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:24.966432 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:24.966454 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:24.968691 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:24.972106 (+ 3415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:24.972113 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:24.972126 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:24.972561 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:24.972840 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:24.972843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:24.972846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:24.973005 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:24.973412 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:24.973417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:24.973721 (+ 304us) client_negotiation.cc:769] Sending connection context
0925 11:00:24.973961 (+ 240us) client_negotiation.cc:240] Negotiation successful
0925 11:00:24.974458 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":340,"threads_started":1}
I20260925 11:00:24.977332 25457 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43781
I20260925 11:00:24.977620 25457 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:24.978511 25457 heartbeater.cc:511] Master 127.24.159.254:43781 requested a full tablet report, sending...
I20260925 11:00:24.980680 25267 ts_manager.cc:194] Registered new tserver with Master: 12a94841b4694a2497d4f952d17a2a19 (127.24.159.193:42969)
I20260925 11:00:24.982064 25267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:55203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:00:24.993621 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43781
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:25.302484 25463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:25.302755 25463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:25.302855 25463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:25.309957 25463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:25.310066 25463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:25.310204 25463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:00:25.319720 25463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43781
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:25.320924 25463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:25.322178 25463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:25.334439 25471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:25.334560 25468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:25.334583 25469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:25.335566 25463 server_base.cc:1034] running on GCE node
I20260925 11:00:25.336145 25463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:25.337047 25463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:25.338284 25463 hybrid_clock.cc:648] HybridClock initialized: now 1790334025338234 us; error 42 us; skew 500 ppm
I20260925 11:00:25.338567 25463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:25 dist-test-slave-trms krb5kdc[25222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334025, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:25.341554 25463 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:00:25.342973 25463 webserver.cc:460] Webserver started at http://127.24.159.194:40567/ using document root <none> and password file <none>
I20260925 11:00:25.343623 25463 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:25.343719 25463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:25.343983 25463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:25.346853 25463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "cccf6ce0a8bf4682805dc6b9df5d6520"
format_stamp: "Formatted at 2026-09-25 11:00:25 on dist-test-slave-trms"
server_key: "4535f095d0ebdc9e88ede5e5cdb3320a"
server_key_iv: "851d95c07867140da30f227da0bf4862"
server_key_version: "encryptionkey@0"
I20260925 11:00:25.347543 25463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "cccf6ce0a8bf4682805dc6b9df5d6520"
format_stamp: "Formatted at 2026-09-25 11:00:25 on dist-test-slave-trms"
server_key: "4535f095d0ebdc9e88ede5e5cdb3320a"
server_key_iv: "851d95c07867140da30f227da0bf4862"
server_key_version: "encryptionkey@0"
I20260925 11:00:25.352533 25463 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:00:25.356639 25478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:25.358163 25463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:00:25.358349 25463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "cccf6ce0a8bf4682805dc6b9df5d6520"
format_stamp: "Formatted at 2026-09-25 11:00:25 on dist-test-slave-trms"
server_key: "4535f095d0ebdc9e88ede5e5cdb3320a"
server_key_iv: "851d95c07867140da30f227da0bf4862"
server_key_version: "encryptionkey@0"
I20260925 11:00:25.358579 25463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:25.404023 25463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:25.408285 25463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:25.408567 25463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:25.409525 25463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:25.410872 25463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:25.410943 25463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:25.411054 25463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:25.411103 25463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:25.438827 25463 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:46757
I20260925 11:00:25.438876 25590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:46757 every 8 connection(s)
I20260925 11:00:25.440044 25463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:00:25.447000 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 25463
I20260925 11:00:25.447193 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:00:25.447654 25215 external_mini_cluster.cc:1428] Setting key 6f1fdabffac1f6b4a2c7cfcfe7991820
Sep 25 11:00:25 dist-test-slave-trms krb5kdc[25222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334025, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:25.459987 25459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:25.442536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:39439 (local address 127.24.159.254:43781)
0925 11:00:25.442711 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:25.442715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:25.443866 (+ 1151us) server_negotiation.cc:407] Connection header received
0925 11:00:25.445596 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:25.445601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:25.445669 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:25.445755 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:25.447481 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:25.448213 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:25.449749 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:25.450156 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:25.450284 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:25.454063 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:25.454070 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:25.454075 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:25.454115 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:25.457146 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:25.457786 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:25.457790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:25.457793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:25.457928 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:25.458297 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:25.458302 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:25.458306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:25.458648 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:25.458786 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:25.459426 (+ 640us) server_negotiation.cc:299] Negotiation successful
0925 11:00:25.459657 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:00:25.460825 25593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:25.443046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43781 (local address 127.24.159.194:39439)
0925 11:00:25.443708 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:25.443750 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:25.445340 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:25.445946 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:25.445955 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:25.446525 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:25.447305 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:25.447332 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:25.448368 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:25.448372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:25.449610 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:25.449622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:25.450400 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:25.450408 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:25.450701 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:25.451376 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:25.451406 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:25.453834 (+ 2428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:25.457267 (+ 3433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:25.457274 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:25.457287 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:25.457688 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:25.458021 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:25.458024 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:25.458026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:25.458214 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:25.458757 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:25.458762 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:25.459068 (+ 306us) client_negotiation.cc:769] Sending connection context
0925 11:00:25.459309 (+ 241us) client_negotiation.cc:240] Negotiation successful
0925 11:00:25.459724 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":258,"threads_started":1}
I20260925 11:00:25.462239 25591 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43781
I20260925 11:00:25.462522 25591 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
I20260925 11:00:25.463320 25591 heartbeater.cc:511] Master 127.24.159.254:43781 requested a full tablet report, sending...
I20260925 11:00:25.464775 25267 ts_manager.cc:194] Registered new tserver with Master: cccf6ce0a8bf4682805dc6b9df5d6520 (127.24.159.194:46757)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
I20260925 11:00:25.465622 25267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:39439
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:00:25.484557 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43781
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:25.826414 25596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:25.826716 25596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:25.826833 25596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:25.834057 25596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:25.834206 25596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:25.834362 25596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:00:25.846393 25596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43781
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:25.847903 25596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:25.849555 25596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:25.867199 25602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:25.867313 25604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:25.867414 25601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:25.873423 25596 server_base.cc:1034] running on GCE node
I20260925 11:00:25.874249 25596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:25.875363 25596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:25.876621 25596 hybrid_clock.cc:648] HybridClock initialized: now 1790334025876551 us; error 60 us; skew 500 ppm
I20260925 11:00:25.876951 25596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:25 dist-test-slave-trms krb5kdc[25222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334025, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:25.880064 25596 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:00:25.881783 25596 webserver.cc:460] Webserver started at http://127.24.159.195:39495/ using document root <none> and password file <none>
I20260925 11:00:25.882512 25596 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:25.882611 25596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:25.882875 25596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:25.885722 25596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "43cb6e2e592b4ad6a44ccbe402784828"
format_stamp: "Formatted at 2026-09-25 11:00:25 on dist-test-slave-trms"
server_key: "15429833bbb17bb6dcb6f3406242c979"
server_key_iv: "b3e36cbb872556262c098e898b6b306e"
server_key_version: "encryptionkey@0"
I20260925 11:00:25.886442 25596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "43cb6e2e592b4ad6a44ccbe402784828"
format_stamp: "Formatted at 2026-09-25 11:00:25 on dist-test-slave-trms"
server_key: "15429833bbb17bb6dcb6f3406242c979"
server_key_iv: "b3e36cbb872556262c098e898b6b306e"
server_key_version: "encryptionkey@0"
I20260925 11:00:25.891417 25596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:00:25.894861 25611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:25.896287 25596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:00:25.896463 25596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "43cb6e2e592b4ad6a44ccbe402784828"
format_stamp: "Formatted at 2026-09-25 11:00:25 on dist-test-slave-trms"
server_key: "15429833bbb17bb6dcb6f3406242c979"
server_key_iv: "b3e36cbb872556262c098e898b6b306e"
server_key_version: "encryptionkey@0"
I20260925 11:00:25.896692 25596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:25.911466 25596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:25.915280 25596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:25.915683 25596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:25.916620 25596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:25.918170 25596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:25.918314 25596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260925 11:00:25.918434 25596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:25.918489 25596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:25.967604 25596 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:36937
I20260925 11:00:25.967636 25723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:36937 every 8 connection(s)
I20260925 11:00:25.969089 25596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:00:25.979122 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 25596
I20260925 11:00:25.979285 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:00:25.979674 25215 external_mini_cluster.cc:1428] Setting key 3f68b219919b519cf69cd96a4868e353
Sep 25 11:00:25 dist-test-slave-trms krb5kdc[25222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334025, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:25.985699 25457 heartbeater.cc:503] Master 127.24.159.254:43781 was elected leader, sending a full tablet report...
I20260925 11:00:25.990002 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:25.971803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:59497 (local address 127.24.159.254:43781)
0925 11:00:25.972275 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:25.972280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:25.973426 (+ 1146us) server_negotiation.cc:407] Connection header received
0925 11:00:25.975342 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:25.975348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:25.975422 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:25.975630 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:25.977116 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:25.978173 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:25.979871 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:25.980295 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:25.980410 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:25.984164 (+ 3754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:25.984169 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:25.984172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:25.984205 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:25.986853 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:25.987599 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:25.987603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:25.987606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:25.987801 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:25.988149 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:25.988152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:25.988154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:25.988507 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:25.988619 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:25.989317 (+ 698us) server_negotiation.cc:299] Negotiation successful
0925 11:00:25.989490 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":206,"threads_started":1}
I20260925 11:00:25.990685 25727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:25.972414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43781 (local address 127.24.159.195:59497)
0925 11:00:25.973153 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:25.973190 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:25.975026 (+ 1836us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:25.975834 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:25.975843 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:25.976370 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:25.976973 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:25.976991 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:25.978341 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:25.978345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:25.979710 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:25.979725 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:25.980524 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:25.980530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:25.980761 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:25.981458 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:25.981481 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:25.983972 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:25.986994 (+ 3022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:25.987001 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:25.987015 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:25.987499 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:25.987892 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:25.987895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:25.987898 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:25.988072 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:25.988598 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:25.988603 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:25.988943 (+ 340us) client_negotiation.cc:769] Sending connection context
0925 11:00:25.989164 (+ 221us) client_negotiation.cc:240] Negotiation successful
0925 11:00:25.989568 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":316,"threads_started":1}
I20260925 11:00:25.992105 25724 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43781
I20260925 11:00:25.992476 25724 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:25.993521 25724 heartbeater.cc:511] Master 127.24.159.254:43781 requested a full tablet report, sending...
I20260925 11:00:25.995301 25267 ts_manager.cc:194] Registered new tserver with Master: 43cb6e2e592b4ad6a44ccbe402784828 (127.24.159.195:36937)
I20260925 11:00:25.996295 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:00:25.996532 25267 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:59497
Sep 25 11:00:26 dist-test-slave-trms krb5kdc[25222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 11:00:26 dist-test-slave-trms krb5kdc[25222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334026, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:00:26.057998 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:26.040432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56744 (local address 127.24.159.193:42969)
0925 11:00:26.041473 (+ 1041us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:26.041484 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:26.041910 (+ 426us) server_negotiation.cc:407] Connection header received
0925 11:00:26.042111 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:26.042121 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:26.042418 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:26.042559 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:26.045121 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.046626 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.048153 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.048957 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.049071 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:26.052798 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:26.052809 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:26.052825 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:26.052859 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:26.055404 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.055998 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.056006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.056008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.056125 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.056494 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.056498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.056501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.056828 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:26.057100 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:26.057413 (+ 313us) server_negotiation.cc:299] Negotiation successful
0925 11:00:26.057652 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":783,"thread_start_us":523,"threads_started":1}
W20260925 11:00:26.059244 25372 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56744
Sep 25 11:00:26 dist-test-slave-trms krb5kdc[25222](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 11:00:26 dist-test-slave-trms krb5kdc[25222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:00:26.108987 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:26.091131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56754 (local address 127.24.159.193:42969)
0925 11:00:26.091365 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:26.091373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:26.091548 (+ 175us) server_negotiation.cc:407] Connection header received
0925 11:00:26.091712 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:26.091716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:26.091813 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:26.091911 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:26.093597 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.094371 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.096061 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.096719 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.096817 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:26.102284 (+ 5467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:26.102290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:26.102294 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:26.102331 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:26.105875 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.106701 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.106707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.106710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.106958 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.107371 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.107378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.107381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.107770 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:26.108094 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:26.108318 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 11:00:26.108555 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260925 11:00:26.110620 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 25329
I20260925 11:00:26.125111 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:26.111561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56768 (local address 127.24.159.193:42969)
0925 11:00:26.111730 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:26.111734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:26.112069 (+ 335us) server_negotiation.cc:407] Connection header received
0925 11:00:26.112174 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:26.112178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:26.112230 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:26.112304 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:26.113665 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.114677 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.116360 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.117084 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.117173 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:26.120138 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:26.120144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:26.120148 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:26.120176 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:26.123026 (+ 2850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.123697 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.123701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.123703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.123813 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.124132 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.124135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.124137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.124355 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:26.124523 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:26.124720 (+ 197us) server_negotiation.cc:299] Negotiation successful
0925 11:00:26.124864 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 11:00:26.125867 25452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:26.206278 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 25329
I20260925 11:00:26.219678 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 25463
Sep 25 11:00:26 dist-test-slave-trms krb5kdc[25222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:00:26.233887 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:26.220496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.24.159.194:46757)
0925 11:00:26.220996 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:26.221004 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:26.221029 (+ 25us) server_negotiation.cc:407] Connection header received
0925 11:00:26.221088 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:26.221094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:26.221318 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:26.221450 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:26.222527 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.223507 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.224606 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.225119 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.225219 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:26.228879 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:26.228891 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:26.228915 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:26.228955 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:26.231657 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.232227 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.232233 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.232235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.232351 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.232640 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.232643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.232645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.232974 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:26.233183 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:26.233392 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 11:00:26.233580 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":258,"threads_started":1}
I20260925 11:00:26.235162 25586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:26.317757 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 25463
I20260925 11:00:26.335755 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 25596
Sep 25 11:00:26 dist-test-slave-trms krb5kdc[25222](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334026, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:00:26.350909 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:26.336711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.24.159.195:36937)
0925 11:00:26.337275 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:26.337282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:26.337310 (+ 28us) server_negotiation.cc:407] Connection header received
0925 11:00:26.337377 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:26.337383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:26.337588 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:26.337725 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:26.338952 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.339941 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.341195 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:26.341743 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:26.341853 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:26.345682 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:26.345694 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:26.345710 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:26.345742 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:26.348234 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.348863 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.348870 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.348872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.348990 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:26.349326 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:26.349330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:26.349332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:26.349665 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:26.349951 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:26.350281 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 11:00:26.350550 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":277,"threads_started":1}
I20260925 11:00:26.352164 25719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:26.434579 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 25596
I20260925 11:00:26.447592 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 25235
I20260925 11:00:26.448702 25297 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:26.550549 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 25235
2026-09-25T11:00:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4009 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:26 dist-test-slave-trms krb5kdc[25746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:26 dist-test-slave-trms krb5kdc[25746](info): set up 2 sockets
Sep 25 11:00:26 dist-test-slave-trms krb5kdc[25746](info): commencing operation
krb5kdc: starting...
W20260925 11:00:27.534394 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.001s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:00:27 dist-test-slave-trms krb5kdc[25746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:00:27.691408 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:42969
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:42969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:27.995152 25759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:27.995421 25759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:27.995550 25759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:28.002602 25759 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:28.002708 25759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:28.002782 25759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:28.002852 25759 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:28.002910 25759 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:28.012534 25759 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:42969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:42969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:28.013840 25759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:28.015161 25759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:28.026237 25767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:28.026165 25764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:28.026216 25765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:28.027051 25759 server_base.cc:1034] running on GCE node
I20260925 11:00:28.027941 25759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:28.029630 25759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:28.030892 25759 hybrid_clock.cc:648] HybridClock initialized: now 1790334028030820 us; error 53 us; skew 500 ppm
I20260925 11:00:28.031304 25759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:28 dist-test-slave-trms krb5kdc[25746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334028, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:28.039278 25759 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:00:28.040689 25759 webserver.cc:460] Webserver started at http://127.24.159.254:39749/ using document root <none> and password file <none>
I20260925 11:00:28.041352 25759 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:28.041446 25759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:28.041709 25759 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:28.044704 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "e08e4072c95846a294ac3cadd2f9e911"
format_stamp: "Formatted at 2026-09-25 11:00:28 on dist-test-slave-trms"
server_key: "46ae8c0c3e0cffed72aa2d40a419e353"
server_key_iv: "799568fb60f86222f71dbed82f0c8389"
server_key_version: "encryptionkey@0"
I20260925 11:00:28.045398 25759 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "e08e4072c95846a294ac3cadd2f9e911"
format_stamp: "Formatted at 2026-09-25 11:00:28 on dist-test-slave-trms"
server_key: "46ae8c0c3e0cffed72aa2d40a419e353"
server_key_iv: "799568fb60f86222f71dbed82f0c8389"
server_key_version: "encryptionkey@0"
I20260925 11:00:28.050417 25759 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260925 11:00:28.053617 25774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:28.054920 25759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:28.055073 25759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "e08e4072c95846a294ac3cadd2f9e911"
format_stamp: "Formatted at 2026-09-25 11:00:28 on dist-test-slave-trms"
server_key: "46ae8c0c3e0cffed72aa2d40a419e353"
server_key_iv: "799568fb60f86222f71dbed82f0c8389"
server_key_version: "encryptionkey@0"
I20260925 11:00:28.055244 25759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:28.066893 25759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:28.070554 25759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:28.070849 25759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:28.087296 25759 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:42969
I20260925 11:00:28.087303 25825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:42969 every 8 connection(s)
I20260925 11:00:28.088611 25759 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:28.092288 25826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:28.094856 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 25759
I20260925 11:00:28.095048 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:00:28.095618 25215 external_mini_cluster.cc:1428] Setting key 6c84a6261426d5c75880076a8e33c979
I20260925 11:00:28.103276 25826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911: Bootstrap starting.
I20260925 11:00:28.106318 25826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:28.107285 25826 log.cc:824] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911: Log is configured to *not* fsync() on all Append() calls
Sep 25 11:00:28 dist-test-slave-trms krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:28.110391 25826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911: No bootstrap required, opened a new log
I20260925 11:00:28.116676 25829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:28.097563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46396 (local address 127.24.159.254:42969)
0925 11:00:28.098334 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:28.098346 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:28.098387 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:00:28.100015 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:28.100037 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:28.100455 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:28.100811 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:28.102253 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:28.103579 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:28.104794 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:28.105463 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:28.105622 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:28.109744 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:28.109752 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:28.109766 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:28.109801 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:28.112683 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:28.113442 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:28.113447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:28.113454 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:28.113616 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:28.113986 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:28.113989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:28.113991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:28.114520 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:28.114724 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:28.115268 (+ 544us) server_negotiation.cc:299] Negotiation successful
0925 11:00:28.115632 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":313,"threads_started":1}
I20260925 11:00:28.118727 25826 raft_consensus.cc:348] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e4072c95846a294ac3cadd2f9e911" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42969 } }
I20260925 11:00:28.119081 25826 raft_consensus.cc:374] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:28.119184 25826 raft_consensus.cc:729] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08e4072c95846a294ac3cadd2f9e911, State: Initialized, Role: FOLLOWER
I20260925 11:00:28.119691 25826 consensus_queue.cc:260] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e4072c95846a294ac3cadd2f9e911" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42969 } }
I20260925 11:00:28.119911 25826 raft_consensus.cc:388] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:28.120025 25826 raft_consensus.cc:482] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:28.120168 25826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:28.122232 25826 raft_consensus.cc:504] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e4072c95846a294ac3cadd2f9e911" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42969 } }
I20260925 11:00:28.122687 25826 leader_election.cc:302] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e08e4072c95846a294ac3cadd2f9e911; no voters:
I20260925 11:00:28.123260 25826 leader_election.cc:288] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:28.123350 25831 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:28.124356 25831 raft_consensus.cc:686] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [term 1 LEADER]: Becoming Leader. State: Replica: e08e4072c95846a294ac3cadd2f9e911, State: Running, Role: LEADER
I20260925 11:00:28.124823 25831 consensus_queue.cc:237] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e4072c95846a294ac3cadd2f9e911" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42969 } }
I20260925 11:00:28.125165 25826 sys_catalog.cc:564] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:28.127219 25833 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08e4072c95846a294ac3cadd2f9e911. Latest consensus state: current_term: 1 leader_uuid: "e08e4072c95846a294ac3cadd2f9e911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e4072c95846a294ac3cadd2f9e911" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42969 } } }
I20260925 11:00:28.127245 25832 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e08e4072c95846a294ac3cadd2f9e911" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08e4072c95846a294ac3cadd2f9e911" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42969 } } }
I20260925 11:00:28.127638 25833 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:28.127638 25832 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:28.128471 25837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:28.134948 25837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:28.145169 25837 catalog_manager.cc:1347] Generated new cluster ID: b37a8763686e46d6964e780ce0ace6cb
I20260925 11:00:28.145284 25837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:28.172463 25837 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:28.173861 25837 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:28.180506 25837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e08e4072c95846a294ac3cadd2f9e911: Generated new TSK 0
I20260925 11:00:28.181644 25837 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:00:28.220062 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42969
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:28.526664 25852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:28.526929 25852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:28.527060 25852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:28.534467 25852 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:28.534595 25852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:28.534759 25852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:00:28.544435 25852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42969
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:28.545706 25852 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:28.547039 25852 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:28.558943 25857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:28.560176 25860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:28.560344 25858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:28.560880 25852 server_base.cc:1034] running on GCE node
I20260925 11:00:28.561414 25852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:28.562265 25852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:28.563453 25852 hybrid_clock.cc:648] HybridClock initialized: now 1790334028563410 us; error 38 us; skew 500 ppm
I20260925 11:00:28.563740 25852 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:28 dist-test-slave-trms krb5kdc[25746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334028, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:28.566592 25852 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:00:28.567973 25852 webserver.cc:460] Webserver started at http://127.24.159.193:41275/ using document root <none> and password file <none>
I20260925 11:00:28.568630 25852 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:28.568758 25852 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:28.569056 25852 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:28.571923 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "c23472667290403da925e8147eda5bbf"
format_stamp: "Formatted at 2026-09-25 11:00:28 on dist-test-slave-trms"
server_key: "ce595b372464a47a4bd711cdaf06dd1e"
server_key_iv: "cde5055eb6d94958cee77992978d1f32"
server_key_version: "encryptionkey@0"
I20260925 11:00:28.572644 25852 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "c23472667290403da925e8147eda5bbf"
format_stamp: "Formatted at 2026-09-25 11:00:28 on dist-test-slave-trms"
server_key: "ce595b372464a47a4bd711cdaf06dd1e"
server_key_iv: "cde5055eb6d94958cee77992978d1f32"
server_key_version: "encryptionkey@0"
I20260925 11:00:28.577666 25852 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 11:00:28.581081 25867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:28.582443 25852 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:28.582587 25852 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "c23472667290403da925e8147eda5bbf"
format_stamp: "Formatted at 2026-09-25 11:00:28 on dist-test-slave-trms"
server_key: "ce595b372464a47a4bd711cdaf06dd1e"
server_key_iv: "cde5055eb6d94958cee77992978d1f32"
server_key_version: "encryptionkey@0"
I20260925 11:00:28.582767 25852 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:28.605715 25852 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:28.609611 25852 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:28.609874 25852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:28.610800 25852 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:28.612036 25852 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:28.612109 25852 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:28.612221 25852 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:28.612272 25852 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:28.640769 25852 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:39583
I20260925 11:00:28.640812 25979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:39583 every 8 connection(s)
I20260925 11:00:28.642023 25852 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:00:28.645254 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 25852
I20260925 11:00:28.645488 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:00:28.646124 25215 external_mini_cluster.cc:1428] Setting key e473711d0e4e8e5061fd3be7852cf734
Sep 25 11:00:28 dist-test-slave-trms krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334028, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
I20260925 11:00:28.662765 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:28.644380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:33769 (local address 127.24.159.254:42969)
0925 11:00:28.644819 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:28.644823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:28.645888 (+ 1065us) server_negotiation.cc:407] Connection header received
0925 11:00:28.648154 (+ 2266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:28.648159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:28.648237 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:28.648346 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:28.649936 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:28.650948 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:28.652467 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:28.653057 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:28.653178 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:28.656759 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:28.656766 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:28.656770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:28.656809 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:28.659969 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:28.660622 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:28.660628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:28.660632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:28.660794 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:28.661150 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:28.661156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:28.661159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:28.661555 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:28.661706 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:28.662327 (+ 621us) server_negotiation.cc:299] Negotiation successful
0925 11:00:28.662496 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":218,"threads_started":1}
I20260925 11:00:28.663765 25983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:28.644929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:42969 (local address 127.24.159.193:33769)
0925 11:00:28.645726 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:28.645768 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:28.647849 (+ 2081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:28.648546 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:28.648554 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:28.649136 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:28.649743 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:28.649761 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:28.651066 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:28.651070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:28.652317 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:28.652328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:28.653358 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:28.653394 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:28.653696 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:28.654377 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:28.654399 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:28.656536 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:28.660090 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:28.660097 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:28.660110 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:28.660511 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:28.660889 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:28.660892 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:28.660894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:28.661052 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:28.661659 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:28.661664 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:28.661989 (+ 325us) client_negotiation.cc:769] Sending connection context
0925 11:00:28.662237 (+ 248us) client_negotiation.cc:240] Negotiation successful
0925 11:00:28.662603 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":289,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
I20260925 11:00:28.665273 25980 heartbeater.cc:347] Connected to a master server at 127.24.159.254:42969
I20260925 11:00:28.665599 25980 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:28.666464 25980 heartbeater.cc:511] Master 127.24.159.254:42969 requested a full tablet report, sending...
I20260925 11:00:28.668682 25791 ts_manager.cc:194] Registered new tserver with Master: c23472667290403da925e8147eda5bbf (127.24.159.193:39583)
I20260925 11:00:28.670099 25791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:33769
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:00:28.684803 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42969
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:28.989521 25986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:28.989821 25986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:28.989967 25986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:28.997082 25986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:28.997210 25986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:28.997370 25986 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:00:29.006978 25986 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42969
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:29.008270 25986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:29.009608 25986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:29.021708 25991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:29.021795 25992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:29.021896 25994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:29.022822 25986 server_base.cc:1034] running on GCE node
I20260925 11:00:29.023438 25986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:29.024315 25986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:29.025499 25986 hybrid_clock.cc:648] HybridClock initialized: now 1790334029025451 us; error 43 us; skew 500 ppm
I20260925 11:00:29.025789 25986 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:29 dist-test-slave-trms krb5kdc[25746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334029, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:29.028820 25986 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:00:29.030138 25986 webserver.cc:460] Webserver started at http://127.24.159.194:36389/ using document root <none> and password file <none>
I20260925 11:00:29.030807 25986 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:29.030920 25986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:29.031195 25986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:29.033941 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "0f151fa0556f4f2babe14a89f1dae7ae"
format_stamp: "Formatted at 2026-09-25 11:00:29 on dist-test-slave-trms"
server_key: "c8220f05089a0fe8c0a6dab3628ac127"
server_key_iv: "cc6e9c98ea8ec99969bfb4faa2b9a2f6"
server_key_version: "encryptionkey@0"
I20260925 11:00:29.034710 25986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "0f151fa0556f4f2babe14a89f1dae7ae"
format_stamp: "Formatted at 2026-09-25 11:00:29 on dist-test-slave-trms"
server_key: "c8220f05089a0fe8c0a6dab3628ac127"
server_key_iv: "cc6e9c98ea8ec99969bfb4faa2b9a2f6"
server_key_version: "encryptionkey@0"
I20260925 11:00:29.039554 25986 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260925 11:00:29.042980 26001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:29.044261 25986 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:00:29.044409 25986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "0f151fa0556f4f2babe14a89f1dae7ae"
format_stamp: "Formatted at 2026-09-25 11:00:29 on dist-test-slave-trms"
server_key: "c8220f05089a0fe8c0a6dab3628ac127"
server_key_iv: "cc6e9c98ea8ec99969bfb4faa2b9a2f6"
server_key_version: "encryptionkey@0"
I20260925 11:00:29.044581 25986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:29.054754 25986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:29.058724 25986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:29.058991 25986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:29.059862 25986 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:29.061066 25986 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:29.061141 25986 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:29.061262 25986 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:29.061314 25986 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:29.090132 25986 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:42923
I20260925 11:00:29.090153 26113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:42923 every 8 connection(s)
I20260925 11:00:29.091400 25986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:00:29.097930 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 25986
I20260925 11:00:29.098115 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:00:29.098542 25215 external_mini_cluster.cc:1428] Setting key e208252f22b025c2ea8cf09948a0eb0d
Sep 25 11:00:29 dist-test-slave-trms krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334029, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:29.111351 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.093862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:56363 (local address 127.24.159.254:42969)
0925 11:00:29.094065 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:29.094071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:29.095305 (+ 1234us) server_negotiation.cc:407] Connection header received
0925 11:00:29.097789 (+ 2484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:29.097796 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:29.097886 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:29.098007 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:29.099880 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.100726 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.102243 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.102658 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.102763 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.105952 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:29.105957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:29.105961 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:29.105991 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:29.108430 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:29.109233 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:29.109236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:29.109238 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:29.109361 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:29.109763 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:29.109766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:29.109768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:29.110005 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:29.110108 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:29.110960 (+ 852us) server_negotiation.cc:299] Negotiation successful
0925 11:00:29.111129 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 11:00:29.112802 26116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.094385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:42969 (local address 127.24.159.194:56363)
0925 11:00:29.095130 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:29.095185 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:29.097467 (+ 2282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:29.098264 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:29.098275 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:29.098962 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:29.099709 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.099732 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.100875 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.100879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.102082 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.102094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.102867 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.102874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.103093 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.103702 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:29.103726 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:29.105744 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:29.108563 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:29.108571 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:29.108586 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:29.109135 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:29.109451 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:29.109456 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:29.109459 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:29.109676 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:29.110119 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:29.110127 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:29.110574 (+ 447us) client_negotiation.cc:769] Sending connection context
0925 11:00:29.110875 (+ 301us) client_negotiation.cc:240] Negotiation successful
0925 11:00:29.111422 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":273,"threads_started":1}
I20260925 11:00:29.114208 26114 heartbeater.cc:347] Connected to a master server at 127.24.159.254:42969
I20260925 11:00:29.114506 26114 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
I20260925 11:00:29.115175 26114 heartbeater.cc:511] Master 127.24.159.254:42969 requested a full tablet report, sending...
I20260925 11:00:29.116592 25791 ts_manager.cc:194] Registered new tserver with Master: 0f151fa0556f4f2babe14a89f1dae7ae (127.24.159.194:42923)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
I20260925 11:00:29.117504 25791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:56363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:00:29.135373 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42969
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:29.446645 26119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:29.446923 26119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:29.447050 26119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:29.454270 26119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:29.454383 26119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:29.454540 26119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:00:29.464355 26119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42969
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:29.465628 26119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:29.466966 26119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:29.479220 26124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:29.479221 26125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:29.479244 26127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:29.480247 26119 server_base.cc:1034] running on GCE node
I20260925 11:00:29.480823 26119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:29.481688 26119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:29.482913 26119 hybrid_clock.cc:648] HybridClock initialized: now 1790334029482862 us; error 37 us; skew 500 ppm
I20260925 11:00:29.483223 26119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:29 dist-test-slave-trms krb5kdc[25746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334029, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:29.486217 26119 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:00:29.487632 26119 webserver.cc:460] Webserver started at http://127.24.159.195:38171/ using document root <none> and password file <none>
I20260925 11:00:29.488245 26119 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:29.488356 26119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:29.488634 26119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:29.491397 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "ab756826dfc74315b940aba82a5c66cf"
format_stamp: "Formatted at 2026-09-25 11:00:29 on dist-test-slave-trms"
server_key: "add417062c517c326001b606e70b1aef"
server_key_iv: "fc9cba5b4f656ffe74413ce42e477efc"
server_key_version: "encryptionkey@0"
I20260925 11:00:29.492091 26119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "ab756826dfc74315b940aba82a5c66cf"
format_stamp: "Formatted at 2026-09-25 11:00:29 on dist-test-slave-trms"
server_key: "add417062c517c326001b606e70b1aef"
server_key_iv: "fc9cba5b4f656ffe74413ce42e477efc"
server_key_version: "encryptionkey@0"
I20260925 11:00:29.496802 26119 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 11:00:29.500046 26134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:29.501295 26119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:00:29.501441 26119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "ab756826dfc74315b940aba82a5c66cf"
format_stamp: "Formatted at 2026-09-25 11:00:29 on dist-test-slave-trms"
server_key: "add417062c517c326001b606e70b1aef"
server_key_iv: "fc9cba5b4f656ffe74413ce42e477efc"
server_key_version: "encryptionkey@0"
I20260925 11:00:29.501621 26119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:29.517311 26119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:29.521293 26119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:29.521540 26119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:29.522496 26119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:29.523766 26119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:29.523844 26119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:29.523953 26119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:29.524000 26119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:29.552587 26119 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:43313
I20260925 11:00:29.552632 26246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:43313 every 8 connection(s)
I20260925 11:00:29.553822 26119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:00:29.559718 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 26119
I20260925 11:00:29.559931 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:00:29.560333 25215 external_mini_cluster.cc:1428] Setting key 87fe3d2c067b56184a2b9c2ccd2130c5
Sep 25 11:00:29 dist-test-slave-trms krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334029, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:29.572335 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.556238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:46665 (local address 127.24.159.254:42969)
0925 11:00:29.556443 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:29.556449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:29.557626 (+ 1177us) server_negotiation.cc:407] Connection header received
0925 11:00:29.559583 (+ 1957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:29.559588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:29.559673 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:29.559777 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:29.561492 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.562464 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.564157 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.564550 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.564640 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.567690 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:29.567695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:29.567698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:29.567729 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:29.569951 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:29.570620 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:29.570624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:29.570626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:29.570739 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:29.571061 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:29.571064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:29.571066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:29.571310 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:29.571421 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:29.571951 (+ 530us) server_negotiation.cc:299] Negotiation successful
0925 11:00:29.572103 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:00:29.573233 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.556773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:42969 (local address 127.24.159.195:46665)
0925 11:00:29.557469 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:29.557508 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:29.559305 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:29.560027 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:29.560039 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:29.560595 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:29.561319 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.561345 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.562613 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.562618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.564038 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.564050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.564710 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.564715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.564910 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.565460 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:29.565481 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:29.567495 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:29.570072 (+ 2577us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:29.570079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:29.570091 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:29.570538 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:29.570815 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:29.570818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:29.570820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:29.570986 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:29.571392 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:29.571398 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:29.571677 (+ 279us) client_negotiation.cc:769] Sending connection context
0925 11:00:29.571876 (+ 199us) client_negotiation.cc:240] Negotiation successful
0925 11:00:29.572215 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":271,"threads_started":1}
I20260925 11:00:29.574891 26247 heartbeater.cc:347] Connected to a master server at 127.24.159.254:42969
I20260925 11:00:29.575225 26247 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:29.576035 26247 heartbeater.cc:511] Master 127.24.159.254:42969 requested a full tablet report, sending...
I20260925 11:00:29.577386 25790 ts_manager.cc:194] Registered new tserver with Master: ab756826dfc74315b940aba82a5c66cf (127.24.159.195:43313)
I20260925 11:00:29.578231 25790 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:46665
I20260925 11:00:29.587546 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:00:29.605717 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.593447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46410 (local address 127.24.159.254:42969)
0925 11:00:29.593603 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:29.593609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:29.594021 (+ 412us) server_negotiation.cc:407] Connection header received
0925 11:00:29.594235 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:29.594240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:29.594309 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:29.594393 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:29.595800 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.596530 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.597798 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.598219 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.598336 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.600446 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:29.600451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:29.600455 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:29.600488 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:29.603186 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:29.603806 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:29.603811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:29.603815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:29.604043 (+ 228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:29.604444 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:29.604448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:29.604451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:29.604780 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:29.604994 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:29.605262 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 11:00:29.605434 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 11:00:29.612205 25790 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:00:29.623001 25790 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:00:29.662310 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.638357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37210 (local address 127.24.159.193:39583)
0925 11:00:29.652660 (+ 14303us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:29.652668 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:29.652699 (+ 31us) server_negotiation.cc:407] Connection header received
0925 11:00:29.652791 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:29.652799 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:29.653051 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:29.653910 (+ 859us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:29.654141 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.655622 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.658700 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.660612 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.661263 (+ 651us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.661462 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:29.661530 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 11:00:29.661745 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14160,"thread_start_us":14037,"threads_started":1}
I20260925 11:00:29.662354 26260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.638284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39583 (local address 127.0.0.1:37210)
0925 11:00:29.650662 (+ 12378us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:29.650725 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:29.650948 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:29.653292 (+ 2344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:29.653300 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:29.653330 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:29.653869 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.653884 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.655808 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.655815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.658519 (+ 2704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.658531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.660913 (+ 2382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.660921 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.661157 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.661219 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 11:00:29.661349 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 11:00:29.661549 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12117,"mutex_wait_us":4,"thread_start_us":12853,"threads_started":1}
I20260925 11:00:29.663918 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.638772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.24.159.194:42923)
0925 11:00:29.652217 (+ 13445us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:29.652228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:29.652700 (+ 472us) server_negotiation.cc:407] Connection header received
0925 11:00:29.652925 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:29.652934 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:29.653201 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:29.653370 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:29.655138 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.656530 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.660104 (+ 3574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.661858 (+ 1754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.662062 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.662260 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:29.663178 (+ 918us) server_negotiation.cc:299] Negotiation successful
0925 11:00:29.663414 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13268,"thread_start_us":13117,"threads_started":1}
I20260925 11:00:29.665530 26263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.653306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:43313 (local address 127.0.0.1:44604)
0925 11:00:29.654759 (+ 1453us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:29.654775 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:29.654901 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:29.655517 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:29.655522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:29.655544 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:29.656438 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.656452 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.658891 (+ 2439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.658897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.661973 (+ 3076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.661987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.664471 (+ 2484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.664478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.664724 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.664748 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 11:00:29.664828 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 11:00:29.664958 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1334,"thread_start_us":1190,"threads_started":1}
I20260925 11:00:29.665674 26264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.653397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.24.159.195:43313)
0925 11:00:29.654488 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:29.654495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:29.655026 (+ 531us) server_negotiation.cc:407] Connection header received
0925 11:00:29.655131 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:29.655139 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:29.655339 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:29.655508 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:29.656646 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.658552 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.662242 (+ 3690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.664191 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.664436 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.664610 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:29.664920 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 11:00:29.665102 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":937,"thread_start_us":785,"threads_started":1}
I20260925 11:00:29.666852 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.638432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:42923 (local address 127.0.0.1:41010)
0925 11:00:29.652577 (+ 14145us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:29.652594 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:29.652719 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:29.653762 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:29.653767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:29.653806 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:29.654973 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.654983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.656690 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.656696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.659935 (+ 3239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.659947 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.662710 (+ 2763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.662717 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.662913 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.662945 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 11:00:29.666062 (+ 3117us) client_negotiation.cc:240] Negotiation successful
0925 11:00:29.666223 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14003,"mutex_wait_us":711,"thread_start_us":13860,"threads_started":1}
I20260925 11:00:29.667274 25915 tablet_service.cc:1467] Processing CreateTablet for tablet 1fee0236a0f844108e90959f9eb43478 (DEFAULT_TABLE table=test-table [id=1200f39413084475956f7a7d908a9839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:29.668838 25915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fee0236a0f844108e90959f9eb43478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:29.670015 26182 tablet_service.cc:1467] Processing CreateTablet for tablet 1fee0236a0f844108e90959f9eb43478 (DEFAULT_TABLE table=test-table [id=1200f39413084475956f7a7d908a9839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:29.671427 26182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fee0236a0f844108e90959f9eb43478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:29.670962 26049 tablet_service.cc:1467] Processing CreateTablet for tablet 1fee0236a0f844108e90959f9eb43478 (DEFAULT_TABLE table=test-table [id=1200f39413084475956f7a7d908a9839]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:29.672219 26049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fee0236a0f844108e90959f9eb43478. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:29.672960 25980 heartbeater.cc:503] Master 127.24.159.254:42969 was elected leader, sending a full tablet report...
I20260925 11:00:29.681259 26265 tablet_bootstrap.cc:492] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf: Bootstrap starting.
I20260925 11:00:29.684443 26265 tablet_bootstrap.cc:654] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:29.686543 26265 log.cc:824] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:29.687826 26266 tablet_bootstrap.cc:492] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf: Bootstrap starting.
I20260925 11:00:29.690164 26267 tablet_bootstrap.cc:492] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae: Bootstrap starting.
I20260925 11:00:29.692324 26265 tablet_bootstrap.cc:492] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf: No bootstrap required, opened a new log
I20260925 11:00:29.692759 26265 ts_tablet_manager.cc:1397] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260925 11:00:29.696085 26267 tablet_bootstrap.cc:654] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:29.698374 26266 tablet_bootstrap.cc:654] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:29.698356 26267 log.cc:824] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:29.700372 26266 log.cc:824] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:29.703023 26267 tablet_bootstrap.cc:492] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae: No bootstrap required, opened a new log
I20260925 11:00:29.703388 26267 ts_tablet_manager.cc:1397] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.007s
I20260925 11:00:29.704107 26266 tablet_bootstrap.cc:492] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf: No bootstrap required, opened a new log
I20260925 11:00:29.704432 26266 ts_tablet_manager.cc:1397] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.000s
I20260925 11:00:29.707746 26265 raft_consensus.cc:348] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } }
I20260925 11:00:29.708354 26265 raft_consensus.cc:374] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:29.708540 26265 raft_consensus.cc:729] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c23472667290403da925e8147eda5bbf, State: Initialized, Role: FOLLOWER
I20260925 11:00:29.709334 26265 consensus_queue.cc:260] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } }
I20260925 11:00:29.711855 26265 ts_tablet_manager.cc:1428] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf: Time spent starting tablet: real 0.019s user 0.008s sys 0.012s
I20260925 11:00:29.717145 26267 raft_consensus.cc:348] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } }
I20260925 11:00:29.717820 26267 raft_consensus.cc:374] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:29.717967 26267 raft_consensus.cc:729] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f151fa0556f4f2babe14a89f1dae7ae, State: Initialized, Role: FOLLOWER
I20260925 11:00:29.718696 26267 consensus_queue.cc:260] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } }
I20260925 11:00:29.719372 26266 raft_consensus.cc:348] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } }
I20260925 11:00:29.720016 26266 raft_consensus.cc:374] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:29.720198 26266 raft_consensus.cc:729] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab756826dfc74315b940aba82a5c66cf, State: Initialized, Role: FOLLOWER
I20260925 11:00:29.720978 26266 consensus_queue.cc:260] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } }
I20260925 11:00:29.721272 26114 heartbeater.cc:503] Master 127.24.159.254:42969 was elected leader, sending a full tablet report...
I20260925 11:00:29.721971 26267 ts_tablet_manager.cc:1428] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260925 11:00:29.722751 26247 heartbeater.cc:503] Master 127.24.159.254:42969 was elected leader, sending a full tablet report...
I20260925 11:00:29.723440 26266 ts_tablet_manager.cc:1428] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf: Time spent starting tablet: real 0.019s user 0.018s sys 0.000s
W20260925 11:00:29.805490 26248 tablet.cc:2369] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:00:29.843447 26115 tablet.cc:2369] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:00:29.894397 25981 tablet.cc:2369] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:00:29.922838 26273 raft_consensus.cc:482] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:29.923244 26273 raft_consensus.cc:504] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } }
I20260925 11:00:29.925479 26273 leader_election.cc:288] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c23472667290403da925e8147eda5bbf (127.24.159.193:39583), 0f151fa0556f4f2babe14a89f1dae7ae (127.24.159.194:42923)
I20260925 11:00:29.933387 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.925914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:37415 (local address 127.24.159.194:42923)
0925 11:00:29.926125 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:29.926132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:29.926619 (+ 487us) server_negotiation.cc:407] Connection header received
0925 11:00:29.926785 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:29.926791 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:29.926851 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:29.926925 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:29.927468 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.928538 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.931128 (+ 2590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.932343 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.932455 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.932507 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:29.933014 (+ 507us) server_negotiation.cc:299] Negotiation successful
0925 11:00:29.933100 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 11:00:29.933513 26249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.926317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:42923 (local address 127.24.159.195:37415)
0925 11:00:29.926487 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:29.926503 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:29.926642 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:29.926964 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:29.926968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:29.926997 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:29.927321 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.927332 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.928656 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.928672 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.930948 (+ 2276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.930961 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.932492 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.932498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.932842 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.932879 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 11:00:29.932998 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 11:00:29.933126 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 11:00:29.934830 26069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fee0236a0f844108e90959f9eb43478" candidate_uuid: "ab756826dfc74315b940aba82a5c66cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" is_pre_election: true
I20260925 11:00:29.935762 26069 raft_consensus.cc:2454] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab756826dfc74315b940aba82a5c66cf in term 0.
I20260925 11:00:29.936358 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.926061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:54817 (local address 127.24.159.193:39583)
0925 11:00:29.926310 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:29.926315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:29.928306 (+ 1991us) server_negotiation.cc:407] Connection header received
0925 11:00:29.928444 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:29.928449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:29.928525 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:29.928620 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:29.929952 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.930942 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.933791 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.935366 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.935546 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.935617 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:29.935924 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 11:00:29.936055 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 11:00:29.936442 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:29.926839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39583 (local address 127.24.159.195:54817)
0925 11:00:29.928205 (+ 1366us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:29.928220 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:29.928337 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:29.928637 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:29.928642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:29.928678 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:29.929720 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.929736 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.931103 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.931107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.933490 (+ 2383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:29.933506 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:29.935601 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:29.935606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:29.935773 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:29.935797 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 11:00:29.935869 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 11:00:29.935967 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1230,"thread_start_us":851,"threads_started":1}
I20260925 11:00:29.936928 26137 leader_election.cc:302] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f151fa0556f4f2babe14a89f1dae7ae, ab756826dfc74315b940aba82a5c66cf; no voters:
I20260925 11:00:29.937426 26273 raft_consensus.cc:2781] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:29.937659 26273 raft_consensus.cc:482] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:29.937810 26273 raft_consensus.cc:3037] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:29.937808 25935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fee0236a0f844108e90959f9eb43478" candidate_uuid: "ab756826dfc74315b940aba82a5c66cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c23472667290403da925e8147eda5bbf" is_pre_election: true
I20260925 11:00:29.938571 25935 raft_consensus.cc:2454] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab756826dfc74315b940aba82a5c66cf in term 0.
I20260925 11:00:29.941476 26273 raft_consensus.cc:504] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } }
I20260925 11:00:29.942250 26273 leader_election.cc:288] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [CANDIDATE]: Term 1 election: Requested vote from peers c23472667290403da925e8147eda5bbf (127.24.159.193:39583), 0f151fa0556f4f2babe14a89f1dae7ae (127.24.159.194:42923)
I20260925 11:00:29.942734 25935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fee0236a0f844108e90959f9eb43478" candidate_uuid: "ab756826dfc74315b940aba82a5c66cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c23472667290403da925e8147eda5bbf"
I20260925 11:00:29.942808 26069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fee0236a0f844108e90959f9eb43478" candidate_uuid: "ab756826dfc74315b940aba82a5c66cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f151fa0556f4f2babe14a89f1dae7ae"
I20260925 11:00:29.942955 25935 raft_consensus.cc:3037] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:29.943049 26069 raft_consensus.cc:3037] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:29.946290 26069 raft_consensus.cc:2454] T 1fee0236a0f844108e90959f9eb43478 P 0f151fa0556f4f2babe14a89f1dae7ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab756826dfc74315b940aba82a5c66cf in term 1.
I20260925 11:00:29.946292 25935 raft_consensus.cc:2454] T 1fee0236a0f844108e90959f9eb43478 P c23472667290403da925e8147eda5bbf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab756826dfc74315b940aba82a5c66cf in term 1.
I20260925 11:00:29.946727 26137 leader_election.cc:302] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f151fa0556f4f2babe14a89f1dae7ae, ab756826dfc74315b940aba82a5c66cf; no voters:
I20260925 11:00:29.947057 26273 raft_consensus.cc:2781] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:29.947856 26273 raft_consensus.cc:686] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [term 1 LEADER]: Becoming Leader. State: Replica: ab756826dfc74315b940aba82a5c66cf, State: Running, Role: LEADER
I20260925 11:00:29.948480 26273 consensus_queue.cc:237] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } }
I20260925 11:00:29.953848 25790 catalog_manager.cc:5495] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf reported cstate change: term changed from 0 to 1, leader changed from <none> to ab756826dfc74315b940aba82a5c66cf (127.24.159.195). New cstate: current_term: 1 leader_uuid: "ab756826dfc74315b940aba82a5c66cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c23472667290403da925e8147eda5bbf" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f151fa0556f4f2babe14a89f1dae7ae" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab756826dfc74315b940aba82a5c66cf" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43313 } health_report { overall_health: HEALTHY } } }
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:00:30.050836 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:30.023999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50744 (local address 127.24.159.193:39583)
0925 11:00:30.024244 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:30.024250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:30.025438 (+ 1188us) server_negotiation.cc:407] Connection header received
0925 11:00:30.025602 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:30.025606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:30.025704 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:30.025787 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:30.029212 (+ 3425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.031406 (+ 2194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.034347 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.035511 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.035661 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:30.042585 (+ 6924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:30.042601 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:30.042619 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:30.042663 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:30.047394 (+ 4731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.048124 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.048134 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.048138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.048424 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.048841 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.048846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.048850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.049415 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:30.049784 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:30.050020 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 11:00:30.050256 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[25746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334030, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334030, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:00:30.098385 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:30.083607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50758 (local address 127.24.159.193:39583)
0925 11:00:30.083803 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:30.083809 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:30.084111 (+ 302us) server_negotiation.cc:407] Connection header received
0925 11:00:30.084293 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:30.084297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:30.084361 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:30.084442 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:30.085704 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.086831 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.088630 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.089379 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.089487 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:30.093471 (+ 3984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:30.093476 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:30.093479 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:30.093514 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:30.096016 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.096623 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.096628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.096630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.096749 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.097085 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.097090 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.097093 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.097349 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:30.097574 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:30.097845 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 11:00:30.098047 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[25746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:00:30.142758 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:30.130734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50762 (local address 127.24.159.193:39583)
0925 11:00:30.130966 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:30.130972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:30.131162 (+ 190us) server_negotiation.cc:407] Connection header received
0925 11:00:30.131306 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:30.131310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:30.131373 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:30.131458 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:30.132582 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.133253 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.134423 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.134832 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.134932 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:30.138329 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:30.138334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:30.138337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:30.138371 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:30.140546 (+ 2175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.141172 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.141176 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.141179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.141293 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.141555 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.141557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.141559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.141841 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:30.142024 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:30.142260 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 11:00:30.142440 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260925 11:00:30.144433 25895 tablet_service.cc:2908] Rejecting scan request for tablet 1fee0236a0f844108e90959f9eb43478: Uninitialized: clean time has not yet been initialized
I20260925 11:00:30.147027 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 25852
I20260925 11:00:30.158748 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:30.147903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.24.159.193:39583)
0925 11:00:30.148097 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:30.148103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:30.148473 (+ 370us) server_negotiation.cc:407] Connection header received
0925 11:00:30.148622 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:30.148628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:30.148711 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:30.148794 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:30.149962 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.150668 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.151838 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.152229 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.152345 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:30.154391 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:30.154396 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:30.154399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:30.154428 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:30.156558 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.157209 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.157213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.157216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.157342 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.157626 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.157629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.157631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.157890 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:30.158091 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:30.158305 (+ 214us) server_negotiation.cc:299] Negotiation successful
0925 11:00:30.158511 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260925 11:00:30.159368 25975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:30.247116 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 25852
I20260925 11:00:30.260957 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 25986
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:00:30.274463 26262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:30.261641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42916 (local address 127.24.159.194:42923)
0925 11:00:30.261793 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:30.261798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:30.261814 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:00:30.261951 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:30.261955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:30.262017 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:30.262100 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:30.263207 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.263923 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.265090 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.265453 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.265538 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:30.269194 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:30.269209 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:30.269225 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:30.269266 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:30.271757 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.272383 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.272390 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.272393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.272525 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.272861 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.272865 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.272867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.273230 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:30.273506 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:30.273796 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 11:00:30.274014 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 11:00:30.275084 26109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:30.356819 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:30.356325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39583 (local address 127.24.159.195:51309)
0925 11:00:30.356536 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:30.356663 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260925 11:00:30.357136 26135 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39583: connect: Connection refused (error 111)
I20260925 11:00:30.358565 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:30.358155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39583 (local address 127.24.159.195:39363)
0925 11:00:30.358334 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:30.358439 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260925 11:00:30.358760 26135 consensus_peers.cc:487] T 1fee0236a0f844108e90959f9eb43478 P ab756826dfc74315b940aba82a5c66cf -> Peer c23472667290403da925e8147eda5bbf (127.24.159.193:39583): Couldn't send request to peer c23472667290403da925e8147eda5bbf. Status: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39583: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 11:00:30.361685 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 25986
I20260925 11:00:30.374625 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 26119
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[25746](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334030, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:00:30.388595 26291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:30.375308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33312 (local address 127.24.159.195:43313)
0925 11:00:30.375774 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:30.375779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:30.375798 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:00:30.375852 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:30.375856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:30.375929 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:30.376019 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:30.377195 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.377873 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.379181 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:30.379591 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:30.379701 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:30.383452 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:30.383466 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:30.383484 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:30.383517 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:30.385957 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.386641 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.386649 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.386651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.386793 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:30.387141 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:30.387145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:30.387147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:30.387481 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:30.387750 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:30.388046 (+ 296us) server_negotiation.cc:299] Negotiation successful
0925 11:00:30.388253 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":223,"threads_started":1}
I20260925 11:00:30.389237 26242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:30.478123 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:30.477722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:42923 (local address 127.24.159.195:55485)
0925 11:00:30.477874 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:30.477960 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.194:42923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:00:30.478174 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 26119
I20260925 11:00:30.491449 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 25759
I20260925 11:00:30.492547 25821 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:30.598800 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 25759
2026-09-25T11:00:30Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4048 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[26296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[26296](info): set up 2 sockets
Sep 25 11:00:30 dist-test-slave-trms krb5kdc[26296](info): commencing operation
krb5kdc: starting...
W20260925 11:00:31.586288 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:00:31 dist-test-slave-trms krb5kdc[26296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:00:31.749226 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37061
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:37061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:32.056298 26309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:32.056568 26309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:32.056689 26309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:32.063684 26309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:32.063784 26309 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 11:00:32.063856 26309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:32.063937 26309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:32.063999 26309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:32.064076 26309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:32.073762 26309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:37061
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:32.075052 26309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:32.076354 26309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:32.087412 26314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:32.087433 26317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:32.087455 26315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:32.088143 26309 server_base.cc:1034] running on GCE node
I20260925 11:00:32.088922 26309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:32.090329 26309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:32.091548 26309 hybrid_clock.cc:648] HybridClock initialized: now 1790334032091483 us; error 44 us; skew 500 ppm
I20260925 11:00:32.091858 26309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:32 dist-test-slave-trms krb5kdc[26296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334032, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:32.094770 26309 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:00:32.096143 26309 webserver.cc:460] Webserver started at http://127.24.159.254:35695/ using document root <none> and password file <none>
I20260925 11:00:32.096827 26309 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:32.096930 26309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:32.097231 26309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:32.100124 26309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "540d9d8c389149cf8fe72b02eba80b49"
format_stamp: "Formatted at 2026-09-25 11:00:32 on dist-test-slave-trms"
server_key: "acf845f865f7d2f19dd644acc9303671"
server_key_iv: "3f532304cadf61173658d3bdedc910f2"
server_key_version: "encryptionkey@0"
I20260925 11:00:32.100832 26309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "540d9d8c389149cf8fe72b02eba80b49"
format_stamp: "Formatted at 2026-09-25 11:00:32 on dist-test-slave-trms"
server_key: "acf845f865f7d2f19dd644acc9303671"
server_key_iv: "3f532304cadf61173658d3bdedc910f2"
server_key_version: "encryptionkey@0"
I20260925 11:00:32.105734 26309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:00:32.109177 26324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:32.110492 26309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:32.110647 26309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "540d9d8c389149cf8fe72b02eba80b49"
format_stamp: "Formatted at 2026-09-25 11:00:32 on dist-test-slave-trms"
server_key: "acf845f865f7d2f19dd644acc9303671"
server_key_iv: "3f532304cadf61173658d3bdedc910f2"
server_key_version: "encryptionkey@0"
I20260925 11:00:32.110841 26309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:32.135421 26309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:32.139392 26309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:32.139643 26309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:32.158635 26309 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:37061
I20260925 11:00:32.158653 26385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:37061 every 8 connection(s)
I20260925 11:00:32.159930 26309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:32.163511 26386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:32.164511 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 26309
I20260925 11:00:32.164752 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:00:32.165546 25215 external_mini_cluster.cc:1428] Setting key 86d26fd24fddf8dbb7fc6e86e31a1c5b
I20260925 11:00:32.177677 26386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49: Bootstrap starting.
Sep 25 11:00:32 dist-test-slave-trms krb5kdc[26296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334031, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:32.180891 26386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:32.182183 26386 log.cc:824] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:32.185313 26386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49: No bootstrap required, opened a new log
I20260925 11:00:32.188165 26389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:32.167539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51946 (local address 127.24.159.254:37061)
0925 11:00:32.168347 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:32.168359 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:32.168401 (+ 42us) server_negotiation.cc:407] Connection header received
0925 11:00:32.169964 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:32.169989 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:32.170447 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:32.170768 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:32.172364 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.173643 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.175536 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.176551 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.176734 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:32.180521 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:32.180532 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:32.180551 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:32.180605 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:32.184024 (+ 3419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:32.184649 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:32.184657 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:32.184665 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:32.184886 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:32.185171 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:32.185176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:32.185179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:32.185848 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:32.186145 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:32.186420 (+ 275us) server_negotiation.cc:299] Negotiation successful
0925 11:00:32.186843 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":351,"threads_started":1}
I20260925 11:00:32.194253 26386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d9d8c389149cf8fe72b02eba80b49" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37061 } }
I20260925 11:00:32.194689 26386 raft_consensus.cc:374] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:32.194768 26386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 540d9d8c389149cf8fe72b02eba80b49, State: Initialized, Role: FOLLOWER
I20260925 11:00:32.195274 26386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d9d8c389149cf8fe72b02eba80b49" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37061 } }
I20260925 11:00:32.195485 26386 raft_consensus.cc:388] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:32.195580 26386 raft_consensus.cc:482] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:32.195708 26386 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:32.198083 26386 raft_consensus.cc:504] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d9d8c389149cf8fe72b02eba80b49" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37061 } }
I20260925 11:00:32.198582 26386 leader_election.cc:302] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 540d9d8c389149cf8fe72b02eba80b49; no voters:
I20260925 11:00:32.199097 26386 leader_election.cc:288] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:32.199227 26391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:32.199874 26391 raft_consensus.cc:686] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [term 1 LEADER]: Becoming Leader. State: Replica: 540d9d8c389149cf8fe72b02eba80b49, State: Running, Role: LEADER
I20260925 11:00:32.200410 26391 consensus_queue.cc:237] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d9d8c389149cf8fe72b02eba80b49" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37061 } }
I20260925 11:00:32.201045 26386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:32.202883 26393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 540d9d8c389149cf8fe72b02eba80b49. Latest consensus state: current_term: 1 leader_uuid: "540d9d8c389149cf8fe72b02eba80b49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d9d8c389149cf8fe72b02eba80b49" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37061 } } }
I20260925 11:00:32.203258 26393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:32.205422 26397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:32.205291 26392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "540d9d8c389149cf8fe72b02eba80b49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "540d9d8c389149cf8fe72b02eba80b49" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37061 } } }
I20260925 11:00:32.205605 26392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:32.210498 26397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:32.220012 26397 catalog_manager.cc:1347] Generated new cluster ID: a80510e7e518487f96f0fbe4eed2ff63
I20260925 11:00:32.220126 26397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:32.249007 26397 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:32.250401 26397 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:32.264586 26397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49: Generated new TSK 0
I20260925 11:00:32.265470 26397 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:00:32.314944 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37061
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:32.624593 26412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:32.624904 26412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:32.625046 26412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:32.632159 26412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:32.632284 26412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:32.632407 26412 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 11:00:32.632519 26412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:00:32.642439 26412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.159.254:37061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:32.643666 26412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:32.644990 26412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:32.657351 26420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:32.657477 26417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:32.657411 26418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:32.658244 26412 server_base.cc:1034] running on GCE node
I20260925 11:00:32.658818 26412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:32.659672 26412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:32.660919 26412 hybrid_clock.cc:648] HybridClock initialized: now 1790334032660826 us; error 58 us; skew 500 ppm
I20260925 11:00:32.661214 26412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:32 dist-test-slave-trms krb5kdc[26296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334032, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:32.664127 26412 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:00:32.665529 26412 webserver.cc:460] Webserver started at http://127.24.159.193:40959/ using document root <none> and password file <none>
I20260925 11:00:32.666224 26412 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:32.666327 26412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:32.666620 26412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:32.669387 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9"
format_stamp: "Formatted at 2026-09-25 11:00:32 on dist-test-slave-trms"
server_key: "f544b849c36387f3ca6777b8385e017f"
server_key_iv: "5b9d2b9f53e99e9fa93c2f773c790a88"
server_key_version: "encryptionkey@0"
I20260925 11:00:32.670120 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9"
format_stamp: "Formatted at 2026-09-25 11:00:32 on dist-test-slave-trms"
server_key: "f544b849c36387f3ca6777b8385e017f"
server_key_iv: "5b9d2b9f53e99e9fa93c2f773c790a88"
server_key_version: "encryptionkey@0"
I20260925 11:00:32.675004 26412 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 11:00:32.678377 26427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:32.679775 26412 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:00:32.679929 26412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9"
format_stamp: "Formatted at 2026-09-25 11:00:32 on dist-test-slave-trms"
server_key: "f544b849c36387f3ca6777b8385e017f"
server_key_iv: "5b9d2b9f53e99e9fa93c2f773c790a88"
server_key_version: "encryptionkey@0"
I20260925 11:00:32.680130 26412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:32.705946 26412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:32.709905 26412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:32.710166 26412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:32.712675 26412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:32.712765 26412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:32.712885 26412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:32.712941 26412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 11:00:32 dist-test-slave-trms krb5kdc[26296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334032, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:32.735687 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:32.713384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:57175 (local address 127.24.159.254:37061)
0925 11:00:32.715018 (+ 1634us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:32.715026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:32.715846 (+ 820us) server_negotiation.cc:407] Connection header received
0925 11:00:32.718633 (+ 2787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:32.718637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:32.718711 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:32.718813 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:32.721085 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.721796 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.724202 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.724736 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.724852 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:32.729405 (+ 4553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:32.729413 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:32.729419 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:32.729466 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:32.732351 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:32.733286 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:32.733289 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:32.733291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:32.733411 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:32.733842 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:32.733845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:32.733847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:32.734116 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:32.734260 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:32.735239 (+ 979us) server_negotiation.cc:299] Negotiation successful
0925 11:00:32.735423 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1492,"thread_start_us":1385,"threads_started":1}
I20260925 11:00:32.737568 26437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:32.714124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37061 (local address 127.24.159.193:57175)
0925 11:00:32.715490 (+ 1366us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:32.715550 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:32.718285 (+ 2735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:32.719142 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:32.719154 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:32.719938 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:32.720893 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:32.720920 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.721953 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.721959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:32.724057 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:32.724071 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.724987 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.724995 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:32.725336 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:32.726093 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:32.726115 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:32.729162 (+ 3047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:32.732495 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:32.732505 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:32.732528 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:32.733163 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:32.733501 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:32.733506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:32.733510 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:32.733748 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:32.734242 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:32.734252 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:32.734791 (+ 539us) client_negotiation.cc:769] Sending connection context
0925 11:00:32.735140 (+ 349us) client_negotiation.cc:240] Negotiation successful
0925 11:00:32.735754 (+ 614us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1056,"thread_start_us":776,"threads_started":1}
I20260925 11:00:32.757755 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:32.744359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:36333 (local address 127.24.159.254:37061)
0925 11:00:32.744554 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:32.744560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:32.745043 (+ 483us) server_negotiation.cc:407] Connection header received
0925 11:00:32.745187 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:32.745192 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:32.745283 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:32.745391 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:32.746237 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.747322 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.749227 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.749782 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.749912 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:32.751865 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:32.751871 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:32.751876 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:32.751917 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:32.755129 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:32.755957 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:32.755962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:32.755966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:32.756093 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:32.756424 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:32.756429 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:32.756433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:32.756698 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:32.756838 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:32.757220 (+ 382us) server_negotiation.cc:299] Negotiation successful
0925 11:00:32.757428 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 11:00:32.757867 26506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:32.744351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37061 (local address 127.24.159.193:36333)
0925 11:00:32.744958 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:32.744976 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:32.745086 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:32.745399 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:32.745403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:32.745806 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:32.746086 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:32.746094 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.747464 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.747472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:32.749032 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:32.749045 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:32.749972 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:32.749978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:32.750147 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:32.750775 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:32.750799 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:32.751641 (+ 842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:32.755284 (+ 3643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:32.755289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:32.755294 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:32.755844 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:32.756194 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:32.756200 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:32.756204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:32.756326 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:32.756808 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:32.756812 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:32.756989 (+ 177us) client_negotiation.cc:769] Sending connection context
0925 11:00:32.757105 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 11:00:32.757440 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":312,"threads_started":1}
I20260925 11:00:32.761670 26412 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:46761
I20260925 11:00:32.761739 26547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:46761 every 8 connection(s)
I20260925 11:00:32.762879 26412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:00:32.764721 26548 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37061
I20260925 11:00:32.764906 26548 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:32.765481 26548 heartbeater.cc:511] Master 127.24.159.254:37061 requested a full tablet report, sending...
I20260925 11:00:32.767339 26341 ts_manager.cc:194] Registered new tserver with Master: 7624c41b23d64c66b22ba6d8c2d2ffb9 (127.24.159.193:46761)
I20260925 11:00:32.768718 26341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:57175
I20260925 11:00:32.772401 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 26412
I20260925 11:00:32.772593 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:00:32.773155 25215 external_mini_cluster.cc:1428] Setting key df6e9263e949add9e04d5d9212742b55
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:00:32.807049 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37061
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:33.113161 26552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:33.113459 26552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:33.113595 26552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:33.120842 26552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:33.120962 26552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:33.121084 26552 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 11:00:33.121196 26552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:00:33.130877 26552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.159.254:37061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:33.132117 26552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:33.133415 26552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:33.145730 26557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:33.145809 26560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:33.145927 26558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:33.146435 26552 server_base.cc:1034] running on GCE node
I20260925 11:00:33.147001 26552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:33.147861 26552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:33.149061 26552 hybrid_clock.cc:648] HybridClock initialized: now 1790334033148989 us; error 47 us; skew 500 ppm
I20260925 11:00:33.149336 26552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:33 dist-test-slave-trms krb5kdc[26296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334033, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:33.156883 26552 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:00:33.158346 26552 webserver.cc:460] Webserver started at http://127.24.159.194:42337/ using document root <none> and password file <none>
I20260925 11:00:33.158998 26552 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:33.159091 26552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:33.159385 26552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:33.162369 26552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "d34431b9d5dc4f8baa3e3a329639be8d"
format_stamp: "Formatted at 2026-09-25 11:00:33 on dist-test-slave-trms"
server_key: "09b26c2bded20ad3f376168a60e4602c"
server_key_iv: "0aa41809b70146dca6d1293ca0c5abea"
server_key_version: "encryptionkey@0"
I20260925 11:00:33.163131 26552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "d34431b9d5dc4f8baa3e3a329639be8d"
format_stamp: "Formatted at 2026-09-25 11:00:33 on dist-test-slave-trms"
server_key: "09b26c2bded20ad3f376168a60e4602c"
server_key_iv: "0aa41809b70146dca6d1293ca0c5abea"
server_key_version: "encryptionkey@0"
I20260925 11:00:33.168089 26552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:00:33.171480 26567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:33.172813 26552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:00:33.172962 26552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "d34431b9d5dc4f8baa3e3a329639be8d"
format_stamp: "Formatted at 2026-09-25 11:00:33 on dist-test-slave-trms"
server_key: "09b26c2bded20ad3f376168a60e4602c"
server_key_iv: "0aa41809b70146dca6d1293ca0c5abea"
server_key_version: "encryptionkey@0"
I20260925 11:00:33.173151 26552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:33.187997 26552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:33.191586 26552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:33.191821 26552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:33.194406 26552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:33.194507 26552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:33.194650 26552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:33.194722 26552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 11:00:33 dist-test-slave-trms krb5kdc[26296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334033, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:33.216722 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.194805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:34147 (local address 127.24.159.254:37061)
0925 11:00:33.195001 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:33.195007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:33.196319 (+ 1312us) server_negotiation.cc:407] Connection header received
0925 11:00:33.198361 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:33.198367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:33.198453 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:33.198567 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:33.200747 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.201672 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.204107 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.204765 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.204920 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.209505 (+ 4585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:33.209513 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:33.209518 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:33.209564 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:33.212621 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.213837 (+ 1216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.213842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.213847 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.214016 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.214663 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.214669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.214673 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.215117 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:33.215284 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:33.216078 (+ 794us) server_negotiation.cc:299] Negotiation successful
0925 11:00:33.216337 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:00:33.217520 26576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.195290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37061 (local address 127.24.159.194:34147)
0925 11:00:33.196140 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:33.196185 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:33.197997 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:33.198831 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:33.198847 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:33.199597 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:33.200490 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.200519 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.201872 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.201881 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.203955 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.203969 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.205033 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.205044 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.205278 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.205960 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:33.205981 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:33.209223 (+ 3242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:33.212829 (+ 3606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:33.212840 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:33.212869 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:33.213658 (+ 789us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:33.214172 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:33.214178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:33.214181 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:33.214490 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:33.215290 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:33.215300 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:33.215690 (+ 390us) client_negotiation.cc:769] Sending connection context
0925 11:00:33.215932 (+ 242us) client_negotiation.cc:240] Negotiation successful
0925 11:00:33.216339 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":329,"threads_started":1}
I20260925 11:00:33.236980 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.223472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:39901 (local address 127.24.159.254:37061)
0925 11:00:33.223679 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:33.223686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:33.224152 (+ 466us) server_negotiation.cc:407] Connection header received
0925 11:00:33.224322 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:33.224326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:33.224396 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:33.224476 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:33.225644 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.226392 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.228304 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.228714 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.228800 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.231272 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:33.231277 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:33.231280 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:33.231311 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:33.233999 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.234971 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.234976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.234979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.235121 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.235484 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.235487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.235489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.235784 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:33.235934 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:33.236389 (+ 455us) server_negotiation.cc:299] Negotiation successful
0925 11:00:33.236629 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 11:00:33.237247 26649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.223417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37061 (local address 127.24.159.194:39901)
0925 11:00:33.224045 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:33.224067 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:33.224219 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:33.224518 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:33.224523 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:33.225143 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:33.225521 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.225533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.226521 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.226527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.228182 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.228197 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.228889 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.228896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.229075 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.229770 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:33.229807 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:33.231085 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:33.234162 (+ 3077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:33.234167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:33.234172 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:33.234845 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:33.235226 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:33.235232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:33.235236 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:33.235396 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:33.235878 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:33.235883 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:33.236109 (+ 226us) client_negotiation.cc:769] Sending connection context
0925 11:00:33.236275 (+ 166us) client_negotiation.cc:240] Negotiation successful
0925 11:00:33.236684 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":291,"threads_started":1}
I20260925 11:00:33.244199 26552 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:38447
I20260925 11:00:33.244237 26686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:38447 every 8 connection(s)
I20260925 11:00:33.245325 26552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:00:33.247115 26687 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37061
I20260925 11:00:33.247329 26687 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:33.247925 26687 heartbeater.cc:511] Master 127.24.159.254:37061 requested a full tablet report, sending...
I20260925 11:00:33.249166 26341 ts_manager.cc:194] Registered new tserver with Master: d34431b9d5dc4f8baa3e3a329639be8d (127.24.159.194:38447)
I20260925 11:00:33.249930 26341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:34147
I20260925 11:00:33.252059 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 26552
I20260925 11:00:33.252233 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:00:33.252691 25215 external_mini_cluster.cc:1428] Setting key 23984601f4f820f9d95c3ca04ace4a06
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:00:33.286718 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37061
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:33.592320 26691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:33.592612 26691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:33.592756 26691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:33.600095 26691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:33.600222 26691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:33.600343 26691 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 11:00:33.600455 26691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:00:33.610154 26691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.159.254:37061
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:33.611402 26691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:33.612689 26691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:33.624948 26696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:33.624990 26697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:33.625013 26699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:33.625797 26691 server_base.cc:1034] running on GCE node
I20260925 11:00:33.626451 26691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:33.627270 26691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:33.628469 26691 hybrid_clock.cc:648] HybridClock initialized: now 1790334033628401 us; error 43 us; skew 500 ppm
I20260925 11:00:33.628751 26691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:33 dist-test-slave-trms krb5kdc[26296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334033, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:33.631632 26691 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:00:33.632973 26691 webserver.cc:460] Webserver started at http://127.24.159.195:44571/ using document root <none> and password file <none>
I20260925 11:00:33.633584 26691 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:33.633667 26691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:33.633931 26691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:33.636719 26691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "4477e9d34c4046bb9fbca0c1b426ec36"
format_stamp: "Formatted at 2026-09-25 11:00:33 on dist-test-slave-trms"
server_key: "ed98a68c69e233bcbb27c3a8d84706d3"
server_key_iv: "2caacd75e64662ad3a3a6d19f2f719cc"
server_key_version: "encryptionkey@0"
I20260925 11:00:33.637449 26691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "4477e9d34c4046bb9fbca0c1b426ec36"
format_stamp: "Formatted at 2026-09-25 11:00:33 on dist-test-slave-trms"
server_key: "ed98a68c69e233bcbb27c3a8d84706d3"
server_key_iv: "2caacd75e64662ad3a3a6d19f2f719cc"
server_key_version: "encryptionkey@0"
I20260925 11:00:33.642395 26691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 11:00:33.645550 26706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:33.646847 26691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:33.647001 26691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "4477e9d34c4046bb9fbca0c1b426ec36"
format_stamp: "Formatted at 2026-09-25 11:00:33 on dist-test-slave-trms"
server_key: "ed98a68c69e233bcbb27c3a8d84706d3"
server_key_iv: "2caacd75e64662ad3a3a6d19f2f719cc"
server_key_version: "encryptionkey@0"
I20260925 11:00:33.647184 26691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:33.669243 26691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:33.673460 26691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:33.673696 26691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:33.676291 26691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:33.676388 26691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:33.676510 26691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:33.676574 26691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 11:00:33 dist-test-slave-trms krb5kdc[26296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334033, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:33.698645 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.677394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:43815 (local address 127.24.159.254:37061)
0925 11:00:33.677561 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:33.677566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:33.678598 (+ 1032us) server_negotiation.cc:407] Connection header received
0925 11:00:33.680613 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:33.680619 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:33.680706 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:33.680800 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:33.682816 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.683831 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.686742 (+ 2911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.687144 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.687241 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.691938 (+ 4697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:33.691945 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:33.691950 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:33.691996 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:33.695417 (+ 3421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.696156 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.696162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.696166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.696335 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.696740 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.696745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.696749 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.697125 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:33.697273 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:33.697964 (+ 691us) server_negotiation.cc:299] Negotiation successful
0925 11:00:33.698233 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 11:00:33.699862 26715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.677608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37061 (local address 127.24.159.195:43815)
0925 11:00:33.678444 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:33.678490 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:33.680330 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:33.681043 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:33.681056 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:33.681719 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:33.682628 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.682659 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.683991 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.684019 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.686590 (+ 2571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.686607 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.687375 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.687383 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.687672 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.688366 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:33.688401 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:33.691673 (+ 3272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:33.695587 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:33.695595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:33.695610 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:33.696035 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:33.696441 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:33.696444 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:33.696446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:33.696644 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:33.697229 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:33.697235 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:33.697603 (+ 368us) client_negotiation.cc:769] Sending connection context
0925 11:00:33.697826 (+ 223us) client_negotiation.cc:240] Negotiation successful
0925 11:00:33.698244 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":319,"threads_started":1}
I20260925 11:00:33.720523 26801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.707167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37061 (local address 127.24.159.195:54663)
0925 11:00:33.707761 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:33.707780 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:33.707901 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:33.708212 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:33.708215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:33.708629 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:33.708925 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.708934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.710446 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.710452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.712192 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.712210 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.713105 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.713116 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.713313 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.713970 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:33.714007 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:33.715026 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:33.717920 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:33.717926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:33.717930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:33.718438 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:33.718797 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:33.718802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:33.718806 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:33.718938 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:33.719474 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:33.719478 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:33.719691 (+ 213us) client_negotiation.cc:769] Sending connection context
0925 11:00:33.719811 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 11:00:33.720089 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":324,"threads_started":1}
I20260925 11:00:33.721485 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.707041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:54663 (local address 127.24.159.254:37061)
0925 11:00:33.707240 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:33.707247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:33.707864 (+ 617us) server_negotiation.cc:407] Connection header received
0925 11:00:33.708008 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:33.708014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:33.708095 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:33.708203 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:33.709048 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.710245 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.712350 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.712892 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.713035 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.715257 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:33.715264 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:33.715269 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:33.715311 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:33.717792 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.718558 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.718563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.718566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.718687 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.719049 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.719055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.719058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.719324 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:33.720820 (+ 1496us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:33.720956 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 11:00:33.721181 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 11:00:33.721480 26691 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:33817
I20260925 11:00:33.721516 26825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:33817 every 8 connection(s)
I20260925 11:00:33.722934 26691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:00:33.725117 26826 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37061
I20260925 11:00:33.725291 26826 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:33.725826 26826 heartbeater.cc:511] Master 127.24.159.254:37061 requested a full tablet report, sending...
I20260925 11:00:33.727039 26341 ts_manager.cc:194] Registered new tserver with Master: 4477e9d34c4046bb9fbca0c1b426ec36 (127.24.159.195:33817)
I20260925 11:00:33.727814 26341 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:43815
I20260925 11:00:33.730882 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 26691
I20260925 11:00:33.731041 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:00:33.731451 25215 external_mini_cluster.cc:1428] Setting key c7b28ca643c81996910de982f26d2cf9
I20260925 11:00:33.733932 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 11:00:33 dist-test-slave-trms krb5kdc[26296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 11:00:33 dist-test-slave-trms krb5kdc[26296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:33.771353 26548 heartbeater.cc:503] Master 127.24.159.254:37061 was elected leader, sending a full tablet report...
I20260925 11:00:33.775965 26436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.762895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.24.159.254:37061)
0925 11:00:33.763044 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:33.763048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:33.763361 (+ 313us) server_negotiation.cc:407] Connection header received
0925 11:00:33.763548 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:33.763553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:33.763619 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:33.763695 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:33.765417 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.766122 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.767396 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.767790 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.767882 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.771372 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:33.771377 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:33.771381 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:33.771414 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:33.773882 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.774477 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.774481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.774483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.774590 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:33.774868 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:33.774870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:33.774872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:33.775126 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:33.775298 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:33.775513 (+ 215us) server_negotiation.cc:299] Negotiation successful
0925 11:00:33.775677 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260925 11:00:33.779704 26341 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:00:33.786557 26341 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:00:33.825562 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.797447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:38447 (local address 127.0.0.1:40028)
0925 11:00:33.816137 (+ 18690us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:33.816179 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:33.816419 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:33.817567 (+ 1148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:33.817575 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:33.817607 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:33.818009 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.818027 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.820545 (+ 2518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.820549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.822898 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.822926 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.824637 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.824644 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.824828 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.824877 (+ 49us) client_negotiation.cc:769] Sending connection context
0925 11:00:33.825008 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 11:00:33.825189 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17215,"thread_start_us":21135,"threads_started":1}
I20260925 11:00:33.826568 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.797668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.24.159.194:38447)
0925 11:00:33.815219 (+ 17551us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:33.815229 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:33.816622 (+ 1393us) server_negotiation.cc:407] Connection header received
0925 11:00:33.816732 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:33.816741 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:33.817117 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:33.817300 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:33.818251 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.820423 (+ 2172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.823066 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.824449 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.824605 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.824778 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:33.826086 (+ 1308us) server_negotiation.cc:299] Negotiation successful
0925 11:00:33.826279 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17383,"thread_start_us":17275,"threads_started":1}
I20260925 11:00:33.826937 26842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.797606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:33817 (local address 127.0.0.1:59108)
0925 11:00:33.818856 (+ 21250us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:33.818873 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:33.819080 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:33.819882 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:33.819887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:33.819911 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:33.820472 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.820484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.822528 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.822534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.824153 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.824164 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.826093 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.826099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.826304 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.826327 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 11:00:33.826404 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 11:00:33.826518 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21048,"mutex_wait_us":1,"spinlock_wait_cycles":10240,"thread_start_us":17641,"threads_started":1}
I20260925 11:00:33.826932 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.798265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.24.159.195:33817)
0925 11:00:33.806681 (+ 8416us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:33.806693 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:33.818976 (+ 12283us) server_negotiation.cc:407] Connection header received
0925 11:00:33.819276 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:33.819284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:33.819544 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:33.819691 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:33.820669 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.822359 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.824300 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.825586 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.825761 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.825952 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:33.826477 (+ 525us) server_negotiation.cc:299] Negotiation successful
0925 11:00:33.826635 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8203,"thread_start_us":7981,"threads_started":1}
I20260925 11:00:33.827000 26839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.797935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46761 (local address 127.0.0.1:35970)
0925 11:00:33.819085 (+ 21150us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:33.819101 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:33.819312 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:33.820034 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:33.820042 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:33.820073 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:33.820485 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.820494 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.822009 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.822014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.824193 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:33.824208 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.826211 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.826218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:33.826373 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.826405 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 11:00:33.826489 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 11:00:33.826604 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21048,"mutex_wait_us":27,"spinlock_wait_cycles":2432,"thread_start_us":20203,"threads_started":1}
I20260925 11:00:33.827526 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:33.798481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35970 (local address 127.24.159.193:46761)
0925 11:00:33.817957 (+ 19476us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:33.817966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:33.819246 (+ 1280us) server_negotiation.cc:407] Connection header received
0925 11:00:33.819501 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:33.819509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:33.819741 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:33.819928 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:33.820673 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.821853 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.824372 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:33.825693 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:33.825865 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:33.826033 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:33.827117 (+ 1084us) server_negotiation.cc:299] Negotiation successful
0925 11:00:33.827270 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19294,"thread_start_us":19149,"threads_started":1}
I20260925 11:00:33.830612 26617 tablet_service.cc:1467] Processing CreateTablet for tablet 7d788a72d2454acb988c34d3b70747c1 (DEFAULT_TABLE table=test-table [id=db31b06cd673402dae8ae3cf152f6479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:33.830621 26756 tablet_service.cc:1467] Processing CreateTablet for tablet 7d788a72d2454acb988c34d3b70747c1 (DEFAULT_TABLE table=test-table [id=db31b06cd673402dae8ae3cf152f6479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:33.831785 26617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d788a72d2454acb988c34d3b70747c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:33.831470 26478 tablet_service.cc:1467] Processing CreateTablet for tablet 7d788a72d2454acb988c34d3b70747c1 (DEFAULT_TABLE table=test-table [id=db31b06cd673402dae8ae3cf152f6479]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:33.831979 26756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d788a72d2454acb988c34d3b70747c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:33.832671 26478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d788a72d2454acb988c34d3b70747c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:33.843019 26845 tablet_bootstrap.cc:492] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: Bootstrap starting.
I20260925 11:00:33.845841 26846 tablet_bootstrap.cc:492] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: Bootstrap starting.
I20260925 11:00:33.846272 26845 tablet_bootstrap.cc:654] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:33.847402 26847 tablet_bootstrap.cc:492] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Bootstrap starting.
I20260925 11:00:33.849043 26845 log.cc:824] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:33.849390 26846 tablet_bootstrap.cc:654] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:33.850473 26847 tablet_bootstrap.cc:654] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:33.850776 26846 log.cc:824] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:33.851608 26847 log.cc:824] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:33.853259 26845 tablet_bootstrap.cc:492] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: No bootstrap required, opened a new log
I20260925 11:00:33.853564 26845 ts_tablet_manager.cc:1397] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260925 11:00:33.854077 26847 tablet_bootstrap.cc:492] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: No bootstrap required, opened a new log
I20260925 11:00:33.854326 26847 ts_tablet_manager.cc:1397] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260925 11:00:33.860877 26846 tablet_bootstrap.cc:492] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: No bootstrap required, opened a new log
I20260925 11:00:33.861166 26846 ts_tablet_manager.cc:1397] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260925 11:00:33.862167 26847 raft_consensus.cc:348] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } }
I20260925 11:00:33.862578 26847 raft_consensus.cc:374] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:33.862701 26847 raft_consensus.cc:729] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7624c41b23d64c66b22ba6d8c2d2ffb9, State: Initialized, Role: FOLLOWER
I20260925 11:00:33.863286 26847 consensus_queue.cc:260] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } }
I20260925 11:00:33.863775 26845 raft_consensus.cc:348] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } }
I20260925 11:00:33.864241 26845 raft_consensus.cc:374] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:33.864367 26845 raft_consensus.cc:729] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d34431b9d5dc4f8baa3e3a329639be8d, State: Initialized, Role: FOLLOWER
I20260925 11:00:33.864943 26847 ts_tablet_manager.cc:1428] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Time spent starting tablet: real 0.011s user 0.001s sys 0.008s
I20260925 11:00:33.865144 26845 consensus_queue.cc:260] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } }
I20260925 11:00:33.866428 26687 heartbeater.cc:503] Master 127.24.159.254:37061 was elected leader, sending a full tablet report...
I20260925 11:00:33.866802 26845 ts_tablet_manager.cc:1428] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260925 11:00:33.870985 26846 raft_consensus.cc:348] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } }
I20260925 11:00:33.871398 26846 raft_consensus.cc:374] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:33.871505 26846 raft_consensus.cc:729] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4477e9d34c4046bb9fbca0c1b426ec36, State: Initialized, Role: FOLLOWER
I20260925 11:00:33.872013 26846 consensus_queue.cc:260] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } }
I20260925 11:00:33.872895 26826 heartbeater.cc:503] Master 127.24.159.254:37061 was elected leader, sending a full tablet report...
I20260925 11:00:33.873353 26846 ts_tablet_manager.cc:1428] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
W20260925 11:00:33.975186 26827 tablet.cc:2369] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:00:33.997125 26688 tablet.cc:2369] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:00:34.015100 26549 tablet.cc:2369] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:00:34.077751 26853 raft_consensus.cc:482] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:34.078054 26853 raft_consensus.cc:504] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } }
I20260925 11:00:34.079710 26853 leader_election.cc:288] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d34431b9d5dc4f8baa3e3a329639be8d (127.24.159.194:38447), 7624c41b23d64c66b22ba6d8c2d2ffb9 (127.24.159.193:46761)
I20260925 11:00:34.086777 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.080118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:49031 (local address 127.24.159.194:38447)
0925 11:00:34.080322 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.080328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.080729 (+ 401us) server_negotiation.cc:407] Connection header received
0925 11:00:34.080814 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.080819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.080898 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.081008 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:34.081530 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.082631 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.084861 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.085989 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.086145 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.086242 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.086456 (+ 214us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.086582 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:00:34.086810 26715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.080135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:38447 (local address 127.24.159.195:49031)
0925 11:00:34.080467 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:34.080491 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:34.080635 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:34.081035 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:34.081039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:34.081073 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:34.081366 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.081379 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.082754 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.082757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.084721 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.084736 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.086124 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.086129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.086302 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.086327 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 11:00:34.086399 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 11:00:34.086523 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"spinlock_wait_cycles":16640}
I20260925 11:00:34.087229 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.080489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:35987 (local address 127.24.159.193:46761)
0925 11:00:34.080671 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.080678 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.081090 (+ 412us) server_negotiation.cc:407] Connection header received
0925 11:00:34.081344 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.081350 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.081424 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.081504 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:34.081935 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.082896 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.085064 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.086369 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.086509 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.086574 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.086939 (+ 365us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.087040 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:00:34.087736 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.080188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46761 (local address 127.24.159.195:35987)
0925 11:00:34.080992 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:34.081008 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:34.081121 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:34.081518 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:34.081521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:34.081540 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:34.081772 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.081779 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.083012 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.083016 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.084937 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.084947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.086567 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.086573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.086739 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.086769 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 11:00:34.087398 (+ 629us) client_negotiation.cc:240] Negotiation successful
0925 11:00:34.087501 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":1,"thread_start_us":479,"threads_started":1}
I20260925 11:00:34.087852 26637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d788a72d2454acb988c34d3b70747c1" candidate_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" is_pre_election: true
I20260925 11:00:34.088414 26637 raft_consensus.cc:2454] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4477e9d34c4046bb9fbca0c1b426ec36 in term 0.
I20260925 11:00:34.088572 26499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d788a72d2454acb988c34d3b70747c1" candidate_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" is_pre_election: true
I20260925 11:00:34.089027 26499 raft_consensus.cc:2454] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4477e9d34c4046bb9fbca0c1b426ec36 in term 0.
I20260925 11:00:34.089121 26709 leader_election.cc:302] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4477e9d34c4046bb9fbca0c1b426ec36, d34431b9d5dc4f8baa3e3a329639be8d; no voters:
I20260925 11:00:34.089527 26853 raft_consensus.cc:2781] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:34.089679 26853 raft_consensus.cc:482] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:34.089780 26853 raft_consensus.cc:3037] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:34.091943 26853 raft_consensus.cc:504] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } }
I20260925 11:00:34.092468 26853 leader_election.cc:288] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [CANDIDATE]: Term 1 election: Requested vote from peers d34431b9d5dc4f8baa3e3a329639be8d (127.24.159.194:38447), 7624c41b23d64c66b22ba6d8c2d2ffb9 (127.24.159.193:46761)
I20260925 11:00:34.092804 26637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d788a72d2454acb988c34d3b70747c1" candidate_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34431b9d5dc4f8baa3e3a329639be8d"
I20260925 11:00:34.092877 26499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d788a72d2454acb988c34d3b70747c1" candidate_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9"
I20260925 11:00:34.092985 26637 raft_consensus.cc:3037] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:34.093055 26499 raft_consensus.cc:3037] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:34.095191 26499 raft_consensus.cc:2454] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4477e9d34c4046bb9fbca0c1b426ec36 in term 1.
I20260925 11:00:34.095194 26637 raft_consensus.cc:2454] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4477e9d34c4046bb9fbca0c1b426ec36 in term 1.
I20260925 11:00:34.095563 26710 leader_election.cc:302] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4477e9d34c4046bb9fbca0c1b426ec36, 7624c41b23d64c66b22ba6d8c2d2ffb9; no voters:
I20260925 11:00:34.095831 26853 raft_consensus.cc:2781] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:34.096225 26853 raft_consensus.cc:686] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 LEADER]: Becoming Leader. State: Replica: 4477e9d34c4046bb9fbca0c1b426ec36, State: Running, Role: LEADER
I20260925 11:00:34.096724 26853 consensus_queue.cc:237] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } }
I20260925 11:00:34.100935 26340 catalog_manager.cc:5495] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4477e9d34c4046bb9fbca0c1b426ec36 (127.24.159.195). New cstate: current_term: 1 leader_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } health_report { overall_health: UNKNOWN } } }
I20260925 11:00:34.127018 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.121643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59118 (local address 127.24.159.195:33817)
0925 11:00:34.121834 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.121840 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.121970 (+ 130us) server_negotiation.cc:407] Connection header received
0925 11:00:34.122291 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.122296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.122372 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.122500 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:34.122851 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.123531 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.124826 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.125242 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.125347 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.125823 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:34.126379 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:34.126577 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.126662 (+ 85us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.126772 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 11:00:34.134943 26637 raft_consensus.cc:1261] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Refusing update from remote peer 4477e9d34c4046bb9fbca0c1b426ec36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:34.134943 26499 raft_consensus.cc:1261] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Refusing update from remote peer 4477e9d34c4046bb9fbca0c1b426ec36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:34.135984 26855 consensus_queue.cc:1035] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:34.136319 26853 consensus_queue.cc:1035] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:34.147037 26858 mvcc.cc:205] Tried to move back new op lower bound from 7333208203806158848 to 7333208203663556608. Current Snapshot: MvccSnapshot[applied={T|T < 7333208203806158848}]
I20260925 11:00:34.154973 26860 mvcc.cc:205] Tried to move back new op lower bound from 7333208203806158848 to 7333208203663556608. Current Snapshot: MvccSnapshot[applied={T|T < 7333208203806158848 or (T in {7333208203806158848})}]
I20260925 11:00:34.169466 26340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:34.169808 26340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:34.173301 26340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49: Sending DeleteTablet for 3 replicas of tablet 7d788a72d2454acb988c34d3b70747c1
I20260925 11:00:34.174434 26756 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d788a72d2454acb988c34d3b70747c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:34 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:59108
I20260925 11:00:34.174489 26617 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d788a72d2454acb988c34d3b70747c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:34 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:40028
I20260925 11:00:34.175341 26868 tablet_replica.cc:330] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: stopping tablet replica
I20260925 11:00:34.175894 26868 raft_consensus.cc:2229] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:00:34.175853 26478 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d788a72d2454acb988c34d3b70747c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:34 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35970
I20260925 11:00:34.176781 26868 raft_consensus.cc:2258] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:34.179035 26340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51956:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:00:34.179589 26340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:00:34.187031 26756 tablet_service.cc:1467] Processing CreateTablet for tablet eb0c2b169019457c92dd721928498bcb (DEFAULT_TABLE table=test-table [id=2b0209183c774cc599e7d0358b534a96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:34.187057 26616 tablet_service.cc:1467] Processing CreateTablet for tablet eb0c2b169019457c92dd721928498bcb (DEFAULT_TABLE table=test-table [id=2b0209183c774cc599e7d0358b534a96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:34.187543 26756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb0c2b169019457c92dd721928498bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:34.187489 26477 tablet_service.cc:1467] Processing CreateTablet for tablet eb0c2b169019457c92dd721928498bcb (DEFAULT_TABLE table=test-table [id=2b0209183c774cc599e7d0358b534a96]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:34.187645 26616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb0c2b169019457c92dd721928498bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:34.188004 26477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb0c2b169019457c92dd721928498bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:34.194044 26845 tablet_bootstrap.cc:492] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: Bootstrap starting.
I20260925 11:00:34.194360 26846 tablet_bootstrap.cc:492] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: Bootstrap starting.
I20260925 11:00:34.194870 26847 tablet_bootstrap.cc:492] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: Bootstrap starting.
I20260925 11:00:34.195699 26845 tablet_bootstrap.cc:654] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:34.195819 26846 tablet_bootstrap.cc:654] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:34.195989 26847 tablet_bootstrap.cc:654] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:34.197791 26846 tablet_bootstrap.cc:492] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: No bootstrap required, opened a new log
I20260925 11:00:34.197932 26846 ts_tablet_manager.cc:1397] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260925 11:00:34.198491 26845 tablet_bootstrap.cc:492] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: No bootstrap required, opened a new log
I20260925 11:00:34.198630 26845 ts_tablet_manager.cc:1397] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260925 11:00:34.198728 26846 raft_consensus.cc:348] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } }
I20260925 11:00:34.199026 26846 raft_consensus.cc:374] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:34.199208 26846 raft_consensus.cc:729] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4477e9d34c4046bb9fbca0c1b426ec36, State: Initialized, Role: FOLLOWER
I20260925 11:00:34.199343 26845 raft_consensus.cc:348] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } }
I20260925 11:00:34.199597 26845 raft_consensus.cc:374] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:34.199532 26846 consensus_queue.cc:260] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } }
I20260925 11:00:34.199887 26845 raft_consensus.cc:729] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d34431b9d5dc4f8baa3e3a329639be8d, State: Initialized, Role: FOLLOWER
I20260925 11:00:34.200068 26846 ts_tablet_manager.cc:1428] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 11:00:34.200278 26845 consensus_queue.cc:260] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } }
I20260925 11:00:34.201756 26845 ts_tablet_manager.cc:1428] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 11:00:34.203819 26869 tablet_replica.cc:330] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: stopping tablet replica
I20260925 11:00:34.204330 26870 tablet_replica.cc:330] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: stopping tablet replica
I20260925 11:00:34.204792 26870 raft_consensus.cc:2229] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:34.205387 26869 raft_consensus.cc:2229] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:34.205870 26869 raft_consensus.cc:2258] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:34.206707 26870 raft_consensus.cc:2258] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:34.208606 26869 ts_tablet_manager.cc:1905] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:34.209095 26868 ts_tablet_manager.cc:1905] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:34.209898 26870 ts_tablet_manager.cc:1905] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:34.216853 26870 ts_tablet_manager.cc:1918] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:00:34.217036 26870 log.cc:1196] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/7d788a72d2454acb988c34d3b70747c1
I20260925 11:00:34.217607 26870 ts_tablet_manager.cc:1939] T 7d788a72d2454acb988c34d3b70747c1 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Deleting consensus metadata
I20260925 11:00:34.218614 26847 tablet_bootstrap.cc:492] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: No bootstrap required, opened a new log
I20260925 11:00:34.218748 26847 ts_tablet_manager.cc:1397] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.009s
I20260925 11:00:34.219389 26847 raft_consensus.cc:348] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } }
I20260925 11:00:34.219575 26847 raft_consensus.cc:374] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:34.219695 26847 raft_consensus.cc:729] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7624c41b23d64c66b22ba6d8c2d2ffb9, State: Initialized, Role: FOLLOWER
I20260925 11:00:34.219949 26847 consensus_queue.cc:260] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } }
I20260925 11:00:34.220554 26847 ts_tablet_manager.cc:1428] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 11:00:34.221591 26868 ts_tablet_manager.cc:1918] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:00:34.221848 26868 log.cc:1196] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/7d788a72d2454acb988c34d3b70747c1
I20260925 11:00:34.222435 26868 ts_tablet_manager.cc:1939] T 7d788a72d2454acb988c34d3b70747c1 P 4477e9d34c4046bb9fbca0c1b426ec36: Deleting consensus metadata
I20260925 11:00:34.224270 26326 catalog_manager.cc:4841] TS 4477e9d34c4046bb9fbca0c1b426ec36 (127.24.159.195:33817): tablet 7d788a72d2454acb988c34d3b70747c1 (table test-table [id=db31b06cd673402dae8ae3cf152f6479]) successfully deleted
I20260925 11:00:34.225447 26869 ts_tablet_manager.cc:1918] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:00:34.225605 26869 log.cc:1196] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/7d788a72d2454acb988c34d3b70747c1
I20260925 11:00:34.225860 26328 catalog_manager.cc:4841] TS 7624c41b23d64c66b22ba6d8c2d2ffb9 (127.24.159.193:46761): tablet 7d788a72d2454acb988c34d3b70747c1 (table test-table [id=db31b06cd673402dae8ae3cf152f6479]) successfully deleted
I20260925 11:00:34.226508 26869 ts_tablet_manager.cc:1939] T 7d788a72d2454acb988c34d3b70747c1 P d34431b9d5dc4f8baa3e3a329639be8d: Deleting consensus metadata
I20260925 11:00:34.228082 26327 catalog_manager.cc:4841] TS d34431b9d5dc4f8baa3e3a329639be8d (127.24.159.194:38447): tablet 7d788a72d2454acb988c34d3b70747c1 (table test-table [id=db31b06cd673402dae8ae3cf152f6479]) successfully deleted
I20260925 11:00:34.231775 26855 raft_consensus.cc:482] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:34.231956 26855 raft_consensus.cc:504] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } }
I20260925 11:00:34.232630 26855 leader_election.cc:288] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7624c41b23d64c66b22ba6d8c2d2ffb9 (127.24.159.193:46761), d34431b9d5dc4f8baa3e3a329639be8d (127.24.159.194:38447)
I20260925 11:00:34.233053 26636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0c2b169019457c92dd721928498bcb" candidate_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" is_pre_election: true
I20260925 11:00:34.233063 26499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0c2b169019457c92dd721928498bcb" candidate_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" is_pre_election: true
I20260925 11:00:34.233273 26636 raft_consensus.cc:2454] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4477e9d34c4046bb9fbca0c1b426ec36 in term 0.
I20260925 11:00:34.233278 26499 raft_consensus.cc:2454] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4477e9d34c4046bb9fbca0c1b426ec36 in term 0.
I20260925 11:00:34.233651 26709 leader_election.cc:302] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4477e9d34c4046bb9fbca0c1b426ec36, d34431b9d5dc4f8baa3e3a329639be8d; no voters:
I20260925 11:00:34.233928 26855 raft_consensus.cc:2781] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:34.234031 26855 raft_consensus.cc:482] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:34.234134 26855 raft_consensus.cc:3037] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:34.236155 26855 raft_consensus.cc:504] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } }
I20260925 11:00:34.236734 26855 leader_election.cc:288] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [CANDIDATE]: Term 1 election: Requested vote from peers 7624c41b23d64c66b22ba6d8c2d2ffb9 (127.24.159.193:46761), d34431b9d5dc4f8baa3e3a329639be8d (127.24.159.194:38447)
I20260925 11:00:34.237170 26636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0c2b169019457c92dd721928498bcb" candidate_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34431b9d5dc4f8baa3e3a329639be8d"
I20260925 11:00:34.237358 26636 raft_consensus.cc:3037] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:34.237372 26499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb0c2b169019457c92dd721928498bcb" candidate_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9"
I20260925 11:00:34.237571 26499 raft_consensus.cc:3037] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:34.239441 26636 raft_consensus.cc:2454] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4477e9d34c4046bb9fbca0c1b426ec36 in term 1.
I20260925 11:00:34.239814 26709 leader_election.cc:302] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4477e9d34c4046bb9fbca0c1b426ec36, d34431b9d5dc4f8baa3e3a329639be8d; no voters:
I20260925 11:00:34.240211 26855 raft_consensus.cc:2781] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:34.240445 26499 raft_consensus.cc:2454] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4477e9d34c4046bb9fbca0c1b426ec36 in term 1.
I20260925 11:00:34.240589 26855 raft_consensus.cc:686] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 LEADER]: Becoming Leader. State: Replica: 4477e9d34c4046bb9fbca0c1b426ec36, State: Running, Role: LEADER
I20260925 11:00:34.240890 26855 consensus_queue.cc:237] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } }
I20260925 11:00:34.243326 26340 catalog_manager.cc:5495] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4477e9d34c4046bb9fbca0c1b426ec36 (127.24.159.195). New cstate: current_term: 1 leader_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } health_report { overall_health: HEALTHY } } }
Sep 25 11:00:34 dist-test-slave-trms krb5kdc[26296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334032, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:34.294339 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.280346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37061 (local address 127.0.0.1:51970)
0925 11:00:34.280961 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:34.280981 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:34.281265 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:34.281890 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:34.281894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:34.282621 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:34.282919 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.282930 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.284244 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.284250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.285216 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.285229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.285939 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.285945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.286103 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.286802 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:34.286818 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:34.289123 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:34.291983 (+ 2860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:34.291996 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:34.292004 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:34.292404 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:34.292725 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:34.292730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:34.292734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:34.292862 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:34.293333 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:34.293341 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:34.293506 (+ 165us) client_negotiation.cc:769] Sending connection context
0925 11:00:34.293647 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 11:00:34.293899 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":309,"threads_started":1}
I20260925 11:00:34.294474 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.280416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.24.159.254:37061)
0925 11:00:34.281474 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.281478 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.281515 (+ 37us) server_negotiation.cc:407] Connection header received
0925 11:00:34.281558 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.281561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.281643 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.281765 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:34.283079 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.284134 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.285334 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.285743 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.285846 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.289325 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:34.289329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:34.289333 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:34.289361 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:34.291869 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.292504 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.292510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.292513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.292630 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.292953 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.292958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.292961 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.293212 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:34.293333 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.293727 (+ 394us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.294045 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":923,"thread_start_us":701,"threads_started":1}
I20260925 11:00:34.299168 26340 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:51970:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 11:00:34.300158 26340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:00:34.307049 26477 tablet_service.cc:1467] Processing CreateTablet for tablet be2ee13a24d74b99adc159b4d46ad548 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbe8e80e326c41bba353e214b9e8f56e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 11:00:34.307152 26756 tablet_service.cc:1467] Processing CreateTablet for tablet be2ee13a24d74b99adc159b4d46ad548 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbe8e80e326c41bba353e214b9e8f56e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 11:00:34.307369 26617 tablet_service.cc:1467] Processing CreateTablet for tablet be2ee13a24d74b99adc159b4d46ad548 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dbe8e80e326c41bba353e214b9e8f56e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 11:00:34.307631 26477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2ee13a24d74b99adc159b4d46ad548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:34.307716 26756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2ee13a24d74b99adc159b4d46ad548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:34.307891 26617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be2ee13a24d74b99adc159b4d46ad548. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:34.314745 26847 tablet_bootstrap.cc:492] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Bootstrap starting.
I20260925 11:00:34.315586 26846 tablet_bootstrap.cc:492] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: Bootstrap starting.
I20260925 11:00:34.316329 26847 tablet_bootstrap.cc:654] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:34.317183 26846 tablet_bootstrap.cc:654] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:34.318535 26845 tablet_bootstrap.cc:492] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: Bootstrap starting.
I20260925 11:00:34.319444 26846 tablet_bootstrap.cc:492] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: No bootstrap required, opened a new log
I20260925 11:00:34.319597 26846 ts_tablet_manager.cc:1397] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260925 11:00:34.320086 26845 tablet_bootstrap.cc:654] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:34.320204 26847 tablet_bootstrap.cc:492] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: No bootstrap required, opened a new log
I20260925 11:00:34.320369 26847 ts_tablet_manager.cc:1397] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260925 11:00:34.320331 26846 raft_consensus.cc:348] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } }
I20260925 11:00:34.320568 26846 raft_consensus.cc:374] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:34.320643 26846 raft_consensus.cc:729] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4477e9d34c4046bb9fbca0c1b426ec36, State: Initialized, Role: FOLLOWER
I20260925 11:00:34.320905 26846 consensus_queue.cc:260] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } }
I20260925 11:00:34.321064 26847 raft_consensus.cc:348] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } }
I20260925 11:00:34.321328 26847 raft_consensus.cc:374] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:34.321395 26847 raft_consensus.cc:729] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7624c41b23d64c66b22ba6d8c2d2ffb9, State: Initialized, Role: FOLLOWER
I20260925 11:00:34.321350 26855 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.321588 26855 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:34.321666 26847 consensus_queue.cc:260] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } }
I20260925 11:00:34.322269 26847 ts_tablet_manager.cc:1428] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260925 11:00:34.322327 26851 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.322690 26851 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:34.323485 26846 ts_tablet_manager.cc:1428] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260925 11:00:34.332180 26845 tablet_bootstrap.cc:492] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: No bootstrap required, opened a new log
I20260925 11:00:34.332377 26845 ts_tablet_manager.cc:1397] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260925 11:00:34.332988 26845 raft_consensus.cc:348] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } }
I20260925 11:00:34.333180 26845 raft_consensus.cc:374] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:34.333254 26845 raft_consensus.cc:729] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d34431b9d5dc4f8baa3e3a329639be8d, State: Initialized, Role: FOLLOWER
I20260925 11:00:34.333483 26845 consensus_queue.cc:260] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } }
I20260925 11:00:34.334230 26845 ts_tablet_manager.cc:1428] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 11:00:34.334111 26852 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.334546 26852 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:34.434350 26851 raft_consensus.cc:482] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:34.434584 26851 raft_consensus.cc:504] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } }
I20260925 11:00:34.436110 26851 leader_election.cc:288] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4477e9d34c4046bb9fbca0c1b426ec36 (127.24.159.195:33817), d34431b9d5dc4f8baa3e3a329639be8d (127.24.159.194:38447)
I20260925 11:00:34.444403 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.436542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:49639 (local address 127.24.159.195:33817)
0925 11:00:34.436721 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.436728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.437958 (+ 1230us) server_negotiation.cc:407] Connection header received
0925 11:00:34.438230 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.438238 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.438317 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.438422 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:34.439088 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.440192 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.442140 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.443355 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.443512 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.443591 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.443948 (+ 357us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.444084 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:00:34.444473 26886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.436754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:33817 (local address 127.24.159.193:49639)
0925 11:00:34.437857 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:34.437873 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:34.438059 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:34.438437 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:34.438441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:34.438462 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:34.438901 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.438912 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.440306 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.440311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.441975 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.441987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.443576 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.443582 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.443762 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.443799 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 11:00:34.443904 (+ 105us) client_negotiation.cc:240] Negotiation successful
0925 11:00:34.444046 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"mutex_wait_us":4,"thread_start_us":804,"threads_started":1}
I20260925 11:00:34.444968 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.436891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:42403 (local address 127.24.159.194:38447)
0925 11:00:34.437062 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.437068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.437714 (+ 646us) server_negotiation.cc:407] Connection header received
0925 11:00:34.438017 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.438022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.438098 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.438210 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:34.438875 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.440002 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.442672 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.443813 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.443951 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.444014 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.444559 (+ 545us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.444702 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:00:34.445788 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.436491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:38447 (local address 127.24.159.193:42403)
0925 11:00:34.437591 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:34.437611 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:34.437832 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:34.438228 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:34.438232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:34.438254 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:34.438683 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.438696 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.440176 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.440182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.442487 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.442502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.444182 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.444189 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.444353 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.444388 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 11:00:34.445135 (+ 747us) client_negotiation.cc:240] Negotiation successful
0925 11:00:34.445276 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":1,"thread_start_us":711,"threads_started":1}
I20260925 11:00:34.445734 26776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2ee13a24d74b99adc159b4d46ad548" candidate_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" is_pre_election: true
I20260925 11:00:34.446280 26776 raft_consensus.cc:2454] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7624c41b23d64c66b22ba6d8c2d2ffb9 in term 0.
I20260925 11:00:34.446511 26636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2ee13a24d74b99adc159b4d46ad548" candidate_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" is_pre_election: true
I20260925 11:00:34.446760 26636 raft_consensus.cc:2454] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7624c41b23d64c66b22ba6d8c2d2ffb9 in term 0.
I20260925 11:00:34.446990 26429 leader_election.cc:302] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4477e9d34c4046bb9fbca0c1b426ec36, 7624c41b23d64c66b22ba6d8c2d2ffb9; no voters:
I20260925 11:00:34.447428 26851 raft_consensus.cc:2781] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:34.447566 26851 raft_consensus.cc:482] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:34.447659 26851 raft_consensus.cc:3037] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:34.449813 26851 raft_consensus.cc:504] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } }
I20260925 11:00:34.450524 26851 leader_election.cc:288] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [CANDIDATE]: Term 1 election: Requested vote from peers 4477e9d34c4046bb9fbca0c1b426ec36 (127.24.159.195:33817), d34431b9d5dc4f8baa3e3a329639be8d (127.24.159.194:38447)
I20260925 11:00:34.451006 26776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2ee13a24d74b99adc159b4d46ad548" candidate_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4477e9d34c4046bb9fbca0c1b426ec36"
I20260925 11:00:34.451056 26636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be2ee13a24d74b99adc159b4d46ad548" candidate_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d34431b9d5dc4f8baa3e3a329639be8d"
I20260925 11:00:34.451217 26776 raft_consensus.cc:3037] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:34.451314 26636 raft_consensus.cc:3037] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:34.454128 26636 raft_consensus.cc:2454] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7624c41b23d64c66b22ba6d8c2d2ffb9 in term 1.
I20260925 11:00:34.454439 26776 raft_consensus.cc:2454] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7624c41b23d64c66b22ba6d8c2d2ffb9 in term 1.
I20260925 11:00:34.454593 26430 leader_election.cc:302] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7624c41b23d64c66b22ba6d8c2d2ffb9, d34431b9d5dc4f8baa3e3a329639be8d; no voters:
I20260925 11:00:34.454900 26851 raft_consensus.cc:2781] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:34.455408 26851 raft_consensus.cc:686] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 LEADER]: Becoming Leader. State: Replica: 7624c41b23d64c66b22ba6d8c2d2ffb9, State: Running, Role: LEADER
I20260925 11:00:34.455804 26851 consensus_queue.cc:237] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } }
I20260925 11:00:34.460671 26887 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: TxnStatusTablet state changed. Reason: New leader 7624c41b23d64c66b22ba6d8c2d2ffb9. Latest consensus state: current_term: 1 leader_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.461207 26887 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: This TxnStatusTablet replica's current role is: LEADER
I20260925 11:00:34.461925 26889 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 11:00:34.462141 26889 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 11:00:34.462667 26341 catalog_manager.cc:5495] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7624c41b23d64c66b22ba6d8c2d2ffb9 (127.24.159.193). New cstate: current_term: 1 leader_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } health_report { overall_health: UNKNOWN } } }
I20260925 11:00:34.485602 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.478110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46761 (local address 127.0.0.1:35972)
0925 11:00:34.478365 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:34.478385 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:34.478516 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:34.479030 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:34.479035 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:34.479049 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:00:34.479236 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.479248 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.480821 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.480827 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.482446 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.482462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.483475 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.483482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.483696 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.483819 (+ 123us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:00:34.485016 (+ 1197us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:00:34.485045 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 11:00:34.485146 (+ 101us) client_negotiation.cc:240] Negotiation successful
0925 11:00:34.485309 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 11:00:34.485677 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.478160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.24.159.193:46761)
0925 11:00:34.478364 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.478370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.478458 (+ 88us) server_negotiation.cc:407] Connection header received
0925 11:00:34.478697 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.478703 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.478814 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.478991 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:34.479375 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.480673 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.482606 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.483316 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.483447 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.484062 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:34.484843 (+ 781us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:34.485000 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.485182 (+ 182us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.485276 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260925 11:00:34.490036 26636 raft_consensus.cc:1261] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Refusing update from remote peer 7624c41b23d64c66b22ba6d8c2d2ffb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:34.490679 26776 raft_consensus.cc:1261] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 FOLLOWER]: Refusing update from remote peer 7624c41b23d64c66b22ba6d8c2d2ffb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:34.491556 26887 consensus_queue.cc:1035] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:34.491894 26851 consensus_queue.cc:1035] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:34.503420 26852 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: TxnStatusTablet state changed. Reason: New leader 7624c41b23d64c66b22ba6d8c2d2ffb9. Latest consensus state: current_term: 1 leader_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.503736 26852 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:34.506773 26855 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: TxnStatusTablet state changed. Reason: New leader 7624c41b23d64c66b22ba6d8c2d2ffb9. Latest consensus state: current_term: 1 leader_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.507094 26855 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:34.508982 26887 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.509250 26887 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: This TxnStatusTablet replica's current role is: LEADER
I20260925 11:00:34.510133 26859 mvcc.cc:205] Tried to move back new op lower bound from 7333208205264084992 to 7333208205134888960. Current Snapshot: MvccSnapshot[applied={T|T < 7333208205264084992 or (T in {7333208205264084992})}]
I20260925 11:00:34.513185 26852 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.513425 26852 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P d34431b9d5dc4f8baa3e3a329639be8d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:34.514840 26851 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.515179 26851 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P 7624c41b23d64c66b22ba6d8c2d2ffb9: This TxnStatusTablet replica's current role is: LEADER
I20260925 11:00:34.516324 26855 tablet_replica.cc:439] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 } } peers { permanent_uuid: "4477e9d34c4046bb9fbca0c1b426ec36" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 33817 } } peers { permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 } } }
I20260925 11:00:34.516633 26855 tablet_replica.cc:442] T be2ee13a24d74b99adc159b4d46ad548 P 4477e9d34c4046bb9fbca0c1b426ec36: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:34.541494 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.533290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:58809 (local address 127.24.159.193:46761)
0925 11:00:34.533704 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.533711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.534250 (+ 539us) server_negotiation.cc:407] Connection header received
0925 11:00:34.534485 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.534490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.534600 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.534718 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:34.535594 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.536820 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.539164 (+ 2344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.539874 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.540020 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.540639 (+ 619us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:34.540794 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:34.540886 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.541087 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.541177 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260925 11:00:34.541618 26896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.533312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46761 (local address 127.24.159.195:58809)
0925 11:00:34.534098 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:34.534130 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:34.534341 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:34.534898 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:34.534904 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:34.534917 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:00:34.535395 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.535409 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.536973 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.536979 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.538982 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.539004 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.540147 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.540157 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.540351 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.540435 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:00:34.540941 (+ 506us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:00:34.540953 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 11:00:34.541040 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 11:00:34.541210 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":419,"threads_started":1}
I20260925 11:00:34.553032 26636 raft_consensus.cc:1261] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Refusing update from remote peer 4477e9d34c4046bb9fbca0c1b426ec36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:34.554116 26499 raft_consensus.cc:1261] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Refusing update from remote peer 4477e9d34c4046bb9fbca0c1b426ec36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:34.554770 26866 consensus_queue.cc:1035] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d34431b9d5dc4f8baa3e3a329639be8d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:34.555421 26855 consensus_queue.cc:1035] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7624c41b23d64c66b22ba6d8c2d2ffb9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:34.617884 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.600902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:39407 (local address 127.24.159.195:33817)
0925 11:00:34.601085 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.601091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.610894 (+ 9803us) server_negotiation.cc:407] Connection header received
0925 11:00:34.611114 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.611120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.611210 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.611316 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:34.612010 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.613165 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.615392 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.616133 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.616267 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.616877 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:34.617027 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:34.617092 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.617471 (+ 379us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.617567 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 11:00:34.619064 26901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.600453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:33817 (local address 127.24.159.193:39407)
0925 11:00:34.610750 (+ 10297us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:34.610771 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:34.610943 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:34.611341 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:34.611346 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:34.611359 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:00:34.611829 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.611842 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.613331 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.613337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.615240 (+ 1903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:34.615258 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.616341 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.616350 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:34.616566 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.616651 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:00:34.617295 (+ 644us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:00:34.617305 (+ 10us) client_negotiation.cc:769] Sending connection context
0925 11:00:34.618426 (+ 1121us) client_negotiation.cc:240] Negotiation successful
0925 11:00:34.618586 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10068,"thread_start_us":9921,"threads_started":1}
I20260925 11:00:34.675959 26341 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:34.676245 26341 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51956:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:34.679637 26341 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 540d9d8c389149cf8fe72b02eba80b49: Sending DeleteTablet for 3 replicas of tablet eb0c2b169019457c92dd721928498bcb
I20260925 11:00:34.680757 26756 tablet_service.cc:1514] Processing DeleteTablet for tablet eb0c2b169019457c92dd721928498bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:34 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:59108
I20260925 11:00:34.680776 26617 tablet_service.cc:1514] Processing DeleteTablet for tablet eb0c2b169019457c92dd721928498bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:34 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:40028
I20260925 11:00:34.681056 26868 tablet_replica.cc:330] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: stopping tablet replica
I20260925 11:00:34.681058 26869 tablet_replica.cc:330] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: stopping tablet replica
I20260925 11:00:34.681113 26477 tablet_service.cc:1514] Processing DeleteTablet for tablet eb0c2b169019457c92dd721928498bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:34 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35970
I20260925 11:00:34.681255 26869 raft_consensus.cc:2229] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:34.681293 26868 raft_consensus.cc:2229] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:00:34.681460 26869 raft_consensus.cc:2258] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:34.681622 26868 raft_consensus.cc:2258] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:34.681895 26870 tablet_replica.cc:330] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: stopping tablet replica
I20260925 11:00:34.682080 26870 raft_consensus.cc:2229] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:34.682293 26870 raft_consensus.cc:2258] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:34.684295 26870 ts_tablet_manager.cc:1905] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:34.687359 26868 ts_tablet_manager.cc:1905] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:34.683117 26869 ts_tablet_manager.cc:1905] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:34.705246 26869 ts_tablet_manager.cc:1918] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:00:34.706373 26869 log.cc:1196] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/eb0c2b169019457c92dd721928498bcb
I20260925 11:00:34.706881 26869 ts_tablet_manager.cc:1939] T eb0c2b169019457c92dd721928498bcb P d34431b9d5dc4f8baa3e3a329639be8d: Deleting consensus metadata
I20260925 11:00:34.708307 26327 catalog_manager.cc:4841] TS d34431b9d5dc4f8baa3e3a329639be8d (127.24.159.194:38447): tablet eb0c2b169019457c92dd721928498bcb (table test-table [id=2b0209183c774cc599e7d0358b534a96]) successfully deleted
I20260925 11:00:34.709651 26870 ts_tablet_manager.cc:1918] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:00:34.709820 26870 log.cc:1196] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/eb0c2b169019457c92dd721928498bcb
I20260925 11:00:34.710281 26868 ts_tablet_manager.cc:1918] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:00:34.710448 26868 log.cc:1196] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/eb0c2b169019457c92dd721928498bcb
I20260925 11:00:34.710853 26868 ts_tablet_manager.cc:1939] T eb0c2b169019457c92dd721928498bcb P 4477e9d34c4046bb9fbca0c1b426ec36: Deleting consensus metadata
I20260925 11:00:34.711905 26326 catalog_manager.cc:4841] TS 4477e9d34c4046bb9fbca0c1b426ec36 (127.24.159.195:33817): tablet eb0c2b169019457c92dd721928498bcb (table test-table [id=2b0209183c774cc599e7d0358b534a96]) successfully deleted
I20260925 11:00:34.712241 26870 ts_tablet_manager.cc:1939] T eb0c2b169019457c92dd721928498bcb P 7624c41b23d64c66b22ba6d8c2d2ffb9: Deleting consensus metadata
I20260925 11:00:34.713335 26328 catalog_manager.cc:4841] TS 7624c41b23d64c66b22ba6d8c2d2ffb9 (127.24.159.193:46761): tablet eb0c2b169019457c92dd721928498bcb (table test-table [id=2b0209183c774cc599e7d0358b534a96]) successfully deleted
Sep 25 11:00:34 dist-test-slave-trms krb5kdc[26296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:00:34.724192 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.708965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35974 (local address 127.24.159.193:46761)
0925 11:00:34.709165 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.709172 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.709568 (+ 396us) server_negotiation.cc:407] Connection header received
0925 11:00:34.709734 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.709740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.709826 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.709937 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:34.711588 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.712441 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.713842 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.714345 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.714452 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.718368 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:34.718381 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:34.718398 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:34.718430 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:34.721387 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.722160 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.722167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.722170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.722349 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.722701 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.722706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.722709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.723073 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:34.723401 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.723631 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.723842 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 11:00:34.724961 26543 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35974
I20260925 11:00:34.746279 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.727943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.24.159.254:37061)
0925 11:00:34.728164 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.728170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.734658 (+ 6488us) server_negotiation.cc:407] Connection header received
0925 11:00:34.734810 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.734815 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.734882 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.734966 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:34.736200 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.737030 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.738250 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.738688 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.738806 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.741190 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:34.741195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:34.741198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:34.741233 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:34.743857 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.744462 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.744467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.744469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.744592 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.744939 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.744944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.744947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.745211 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:34.745460 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.745749 (+ 289us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.745958 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260925 11:00:34.746842 26341 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51974
I20260925 11:00:34.748631 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 26412
I20260925 11:00:34.769856 26844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.749682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.24.159.193:46761)
0925 11:00:34.750715 (+ 1033us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.750722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.756383 (+ 5661us) server_negotiation.cc:407] Connection header received
0925 11:00:34.756570 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.756575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.756646 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.756735 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:34.757952 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.758764 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.759966 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.760424 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.760537 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.762901 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:34.762907 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:34.762912 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:34.762954 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:34.766609 (+ 3655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.767433 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.767439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.767443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.767600 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.767996 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.768001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.768004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.768476 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:34.768721 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.769227 (+ 506us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.769491 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861}
W20260925 11:00:34.771072 26543 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35990
W20260925 11:00:34.771721 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 26412: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:34.771935 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 26412
I20260925 11:00:34.787717 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 26552
Sep 25 11:00:34 dist-test-slave-trms krb5kdc[26296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:00:34.800979 26841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.788404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40040 (local address 127.24.159.194:38447)
0925 11:00:34.788574 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.788579 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.788597 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:00:34.788738 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.788742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.788806 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.788885 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:34.789924 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.790671 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.791812 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.792183 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.792293 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.795970 (+ 3677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:34.795984 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:34.796000 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:34.796037 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:34.798714 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.799281 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.799294 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.799297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.799411 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.799761 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.799764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.799766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.800088 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:34.800308 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.800569 (+ 261us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.800721 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260925 11:00:34.801636 26682 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40040
W20260925 11:00:34.802279 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 26552: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:34.802433 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 26552
I20260925 11:00:34.815982 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 26691
Sep 25 11:00:34 dist-test-slave-trms krb5kdc[26296](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:00:34.829759 26843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:34.816699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59132 (local address 127.24.159.195:33817)
0925 11:00:34.816877 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:34.816883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:34.816902 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:00:34.817054 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:34.817060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:34.817138 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:34.817228 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:34.818427 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.819114 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.820293 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:34.820718 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:34.820826 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:34.824780 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:34.824807 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:34.824823 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:34.824857 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:34.827502 (+ 2645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.828076 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.828082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.828084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.828204 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:34.828553 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:34.828556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:34.828558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:34.828909 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:34.829123 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:34.829366 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 11:00:34.829521 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260925 11:00:34.830428 26821 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59132
W20260925 11:00:34.831004 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 26691: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:34.831132 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 26691
I20260925 11:00:34.847886 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 26309
I20260925 11:00:34.849006 26381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:34.961760 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 26309
2026-09-25T11:00:34Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4370 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:35 dist-test-slave-trms krb5kdc[26911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:35 dist-test-slave-trms krb5kdc[26911](info): set up 2 sockets
Sep 25 11:00:35 dist-test-slave-trms krb5kdc[26911](info): commencing operation
krb5kdc: starting...
W20260925 11:00:35.958535 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:00:36 dist-test-slave-trms krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:00:36.133711 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:41643
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:40557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:41643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:36.440647 26924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:36.440925 26924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:36.441049 26924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:36.448293 26924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:36.448392 26924 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 11:00:36.448437 26924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:36.448520 26924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:36.448577 26924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:36.448659 26924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:36.458473 26924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:41643
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:41643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:36.459718 26924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:36.460984 26924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:36.472196 26929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:36.472137 26930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:36.472172 26932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:36.473371 26924 server_base.cc:1034] running on GCE node
I20260925 11:00:36.474110 26924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:36.475560 26924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:36.476796 26924 hybrid_clock.cc:648] HybridClock initialized: now 1790334036476709 us; error 49 us; skew 500 ppm
I20260925 11:00:36.477104 26924 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:36 dist-test-slave-trms krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334036, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:36.480126 26924 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:00:36.481545 26924 webserver.cc:460] Webserver started at http://127.24.159.254:34797/ using document root <none> and password file <none>
I20260925 11:00:36.482230 26924 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:36.482323 26924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:36.482584 26924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:36.485379 26924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "1bc8152db208416aa6e44d9e42fb1043"
format_stamp: "Formatted at 2026-09-25 11:00:36 on dist-test-slave-trms"
server_key: "d073be70b075322bd62cd8d791657ec4"
server_key_iv: "5fac85cddea73c5d2fde3091d18ae462"
server_key_version: "encryptionkey@0"
I20260925 11:00:36.486117 26924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "1bc8152db208416aa6e44d9e42fb1043"
format_stamp: "Formatted at 2026-09-25 11:00:36 on dist-test-slave-trms"
server_key: "d073be70b075322bd62cd8d791657ec4"
server_key_iv: "5fac85cddea73c5d2fde3091d18ae462"
server_key_version: "encryptionkey@0"
I20260925 11:00:36.490962 26924 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 11:00:36.494122 26939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:36.495393 26924 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:00:36.495551 26924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "1bc8152db208416aa6e44d9e42fb1043"
format_stamp: "Formatted at 2026-09-25 11:00:36 on dist-test-slave-trms"
server_key: "d073be70b075322bd62cd8d791657ec4"
server_key_iv: "5fac85cddea73c5d2fde3091d18ae462"
server_key_version: "encryptionkey@0"
I20260925 11:00:36.495734 26924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:36.518914 26924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:36.522657 26924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:36.522913 26924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:36.542126 26924 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:41643
I20260925 11:00:36.542151 27000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:41643 every 8 connection(s)
I20260925 11:00:36.543469 26924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:36.547084 27001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:36.549150 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 26924
I20260925 11:00:36.549335 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:00:36.550087 25215 external_mini_cluster.cc:1428] Setting key fa59945a9a5f1801fc06f2fdbb4f54ee
I20260925 11:00:36.558935 27001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043: Bootstrap starting.
I20260925 11:00:36.562641 27001 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043: Neither blocks nor log segments found. Creating new log.
Sep 25 11:00:36 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334036, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:36.563807 27001 log.cc:824] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:36.567075 27001 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043: No bootstrap required, opened a new log
I20260925 11:00:36.572954 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:36.552077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60950 (local address 127.24.159.254:41643)
0925 11:00:36.552850 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:36.552862 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:36.552905 (+ 43us) server_negotiation.cc:407] Connection header received
0925 11:00:36.554530 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:36.554552 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:36.554923 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:36.555230 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:36.556829 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:36.558077 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:36.559734 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:36.560393 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:36.560527 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:36.564869 (+ 4342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:36.564875 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:36.564890 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:36.564936 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:36.568463 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:36.569149 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:36.569158 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:36.569166 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:36.569364 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:36.569673 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:36.569678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:36.569684 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:36.570452 (+ 768us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:36.570752 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:36.571157 (+ 405us) server_negotiation.cc:299] Negotiation successful
0925 11:00:36.571593 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":355,"threads_started":1}
I20260925 11:00:36.576642 27001 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc8152db208416aa6e44d9e42fb1043" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41643 } }
I20260925 11:00:36.576965 27001 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:36.577039 27001 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bc8152db208416aa6e44d9e42fb1043, State: Initialized, Role: FOLLOWER
I20260925 11:00:36.577517 27001 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc8152db208416aa6e44d9e42fb1043" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41643 } }
I20260925 11:00:36.577720 27001 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:36.577836 27001 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:36.577961 27001 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:36.580271 27001 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc8152db208416aa6e44d9e42fb1043" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41643 } }
I20260925 11:00:36.580734 27001 leader_election.cc:302] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1bc8152db208416aa6e44d9e42fb1043; no voters:
I20260925 11:00:36.581224 27001 leader_election.cc:288] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:36.581351 27006 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:36.581956 27006 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [term 1 LEADER]: Becoming Leader. State: Replica: 1bc8152db208416aa6e44d9e42fb1043, State: Running, Role: LEADER
I20260925 11:00:36.582508 27006 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc8152db208416aa6e44d9e42fb1043" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41643 } }
I20260925 11:00:36.583184 27001 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:36.585397 27008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1bc8152db208416aa6e44d9e42fb1043. Latest consensus state: current_term: 1 leader_uuid: "1bc8152db208416aa6e44d9e42fb1043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc8152db208416aa6e44d9e42fb1043" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41643 } } }
I20260925 11:00:36.585602 27007 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1bc8152db208416aa6e44d9e42fb1043" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1bc8152db208416aa6e44d9e42fb1043" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41643 } } }
I20260925 11:00:36.585875 27008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:36.585893 27007 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:36.587000 27015 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:36.594081 27015 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:36.602442 27015 catalog_manager.cc:1347] Generated new cluster ID: 1a7c1c52aec34251acc49fab4adc7d49
I20260925 11:00:36.602559 27015 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:36.611696 27015 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:36.612933 27015 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:36.621902 27015 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043: Generated new TSK 0
I20260925 11:00:36.622754 27015 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:00:36.667369 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:41643
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:40557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:36.981662 27027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:36.981976 27027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:36.982116 27027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:36.989537 27027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:36.989660 27027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:36.989778 27027 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 11:00:36.989845 27027 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 11:00:36.989946 27027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:00:36.999801 27027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.159.254:41643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:37.001071 27027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:37.002444 27027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:37.014705 27033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:37.014706 27035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:37.014889 27032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:37.015506 27027 server_base.cc:1034] running on GCE node
I20260925 11:00:37.016068 27027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:37.016924 27027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:37.018098 27027 hybrid_clock.cc:648] HybridClock initialized: now 1790334037018052 us; error 36 us; skew 500 ppm
I20260925 11:00:37.018416 27027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:37 dist-test-slave-trms krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:37.021267 27027 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:00:37.022691 27027 webserver.cc:460] Webserver started at http://127.24.159.193:33745/ using document root <none> and password file <none>
I20260925 11:00:37.023330 27027 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:37.023442 27027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:37.023752 27027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:37.026692 27027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "13eb3805718c4b1c81d183e40ae3c8f1"
format_stamp: "Formatted at 2026-09-25 11:00:37 on dist-test-slave-trms"
server_key: "a14d2de47c2865c486f400dedccfefdb"
server_key_iv: "feb42420c0ae852aca359560df23a9c7"
server_key_version: "encryptionkey@0"
I20260925 11:00:37.027413 27027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "13eb3805718c4b1c81d183e40ae3c8f1"
format_stamp: "Formatted at 2026-09-25 11:00:37 on dist-test-slave-trms"
server_key: "a14d2de47c2865c486f400dedccfefdb"
server_key_iv: "feb42420c0ae852aca359560df23a9c7"
server_key_version: "encryptionkey@0"
I20260925 11:00:37.032316 27027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:00:37.035657 27042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:37.036900 27027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:00:37.037042 27027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "13eb3805718c4b1c81d183e40ae3c8f1"
format_stamp: "Formatted at 2026-09-25 11:00:37 on dist-test-slave-trms"
server_key: "a14d2de47c2865c486f400dedccfefdb"
server_key_iv: "feb42420c0ae852aca359560df23a9c7"
server_key_version: "encryptionkey@0"
I20260925 11:00:37.037213 27027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:37.051258 27027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:37.054975 27027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:37.055229 27027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:37.057816 27027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:37.057931 27027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:37.058053 27027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:37.058125 27027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 11:00:37 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:37.081413 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:37.058908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41643 (local address 127.24.159.193:58781)
0925 11:00:37.059663 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:37.059700 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:37.061532 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:37.062488 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:37.062500 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:37.063253 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:37.064134 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:37.064169 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.065550 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.065555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:37.067745 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:37.067761 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.068615 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.068623 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:37.068879 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:37.069630 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:37.069658 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:37.072868 (+ 3210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:37.076421 (+ 3553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:37.076434 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:37.076453 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:37.077097 (+ 644us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:37.077543 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:37.077548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:37.077552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:37.077806 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:37.078543 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:37.078551 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:37.079024 (+ 473us) client_negotiation.cc:769] Sending connection context
0925 11:00:37.079272 (+ 248us) client_negotiation.cc:240] Negotiation successful
0925 11:00:37.079803 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":320,"threads_started":1}
I20260925 11:00:37.082554 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:37.058278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:58781 (local address 127.24.159.254:41643)
0925 11:00:37.058448 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:37.058456 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:37.059823 (+ 1367us) server_negotiation.cc:407] Connection header received
0925 11:00:37.061907 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:37.061914 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:37.062011 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:37.062134 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:37.064395 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.065396 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.067934 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.068364 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.068487 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:37.073153 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:37.073160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:37.073165 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:37.073206 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:37.076239 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:37.077223 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:37.077229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:37.077233 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:37.077389 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:37.077969 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:37.077973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:37.077976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:37.078359 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:37.081914 (+ 3555us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:37.082031 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 11:00:37.082226 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 11:00:37.101212 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:37.087947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:55837 (local address 127.24.159.254:41643)
0925 11:00:37.088087 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:37.088091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:37.088680 (+ 589us) server_negotiation.cc:407] Connection header received
0925 11:00:37.088845 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:37.088849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:37.088908 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:37.088983 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:37.089886 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.090692 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.092418 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.093038 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.093164 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:37.095517 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:37.095522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:37.095525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:37.095554 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:37.098510 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:37.099317 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:37.099322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:37.099326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:37.099469 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:37.099798 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:37.099803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:37.099807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:37.100105 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:37.100251 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:37.100623 (+ 372us) server_negotiation.cc:299] Negotiation successful
0925 11:00:37.100866 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 11:00:37.101291 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:37.088023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41643 (local address 127.24.159.193:55837)
0925 11:00:37.088573 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:37.088588 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:37.088725 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:37.089023 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:37.089027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:37.089473 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:37.089753 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:37.089762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.090859 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.090863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:37.092272 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:37.092285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.093286 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.093293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:37.093421 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:37.094038 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:37.094054 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:37.095276 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:37.098669 (+ 3393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:37.098675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:37.098680 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:37.099208 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:37.099572 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:37.099577 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:37.099581 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:37.099703 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:37.100213 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:37.100217 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:37.100394 (+ 177us) client_negotiation.cc:769] Sending connection context
0925 11:00:37.100519 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 11:00:37.100833 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":288,"threads_started":1}
I20260925 11:00:37.107201 27027 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:33939
I20260925 11:00:37.107230 27161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:33939 every 8 connection(s)
I20260925 11:00:37.108350 27027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:00:37.110176 27162 heartbeater.cc:347] Connected to a master server at 127.24.159.254:41643
I20260925 11:00:37.110414 27162 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:37.111032 27162 heartbeater.cc:511] Master 127.24.159.254:41643 requested a full tablet report, sending...
I20260925 11:00:37.112856 26956 ts_manager.cc:194] Registered new tserver with Master: 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939)
I20260925 11:00:37.114283 26956 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:58781
I20260925 11:00:37.115449 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 27027
I20260925 11:00:37.115650 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:00:37.116454 25215 external_mini_cluster.cc:1428] Setting key 8b6707ce56024feeacde2af4f6e5c5f1
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:00:37.151153 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:41643
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:40557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:37.453426 27166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:37.453678 27166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:37.453810 27166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:37.460798 27166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:37.460904 27166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:37.461033 27166 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 11:00:37.461092 27166 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 11:00:37.461197 27166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:00:37.470707 27166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.159.254:41643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:37.471902 27166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:37.473156 27166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:37.485327 27171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:37.485364 27174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:37.485296 27172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:37.486311 27166 server_base.cc:1034] running on GCE node
I20260925 11:00:37.486876 27166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:37.487802 27166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:37.488991 27166 hybrid_clock.cc:648] HybridClock initialized: now 1790334037488944 us; error 41 us; skew 500 ppm
I20260925 11:00:37.489293 27166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:37 dist-test-slave-trms krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:37.492079 27166 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:00:37.493399 27166 webserver.cc:460] Webserver started at http://127.24.159.194:34395/ using document root <none> and password file <none>
I20260925 11:00:37.494014 27166 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:37.494122 27166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:37.494490 27166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:37.497216 27166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "dafd670fbd634028b7f3b5f14401f160"
format_stamp: "Formatted at 2026-09-25 11:00:37 on dist-test-slave-trms"
server_key: "093097dc2ba3548c0bcc0f8729cca283"
server_key_iv: "e6f2bc82fbac6b8a49a6ca4c67a4be51"
server_key_version: "encryptionkey@0"
I20260925 11:00:37.497896 27166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "dafd670fbd634028b7f3b5f14401f160"
format_stamp: "Formatted at 2026-09-25 11:00:37 on dist-test-slave-trms"
server_key: "093097dc2ba3548c0bcc0f8729cca283"
server_key_iv: "e6f2bc82fbac6b8a49a6ca4c67a4be51"
server_key_version: "encryptionkey@0"
I20260925 11:00:37.502730 27166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 11:00:37.505980 27181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:37.507261 27166 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:37.507417 27166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "dafd670fbd634028b7f3b5f14401f160"
format_stamp: "Formatted at 2026-09-25 11:00:37 on dist-test-slave-trms"
server_key: "093097dc2ba3548c0bcc0f8729cca283"
server_key_iv: "e6f2bc82fbac6b8a49a6ca4c67a4be51"
server_key_version: "encryptionkey@0"
I20260925 11:00:37.507591 27166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:37.525324 27166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:37.529251 27166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:37.529526 27166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:37.532169 27166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:37.532285 27166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:37.532414 27166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:37.532495 27166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 11:00:37 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:37.557161 27190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:37.533248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41643 (local address 127.24.159.194:55123)
0925 11:00:37.534083 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:37.534123 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:37.536140 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:37.536876 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:37.536887 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:37.537581 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:37.538513 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:37.538543 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.540053 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.540058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:37.542231 (+ 2173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:37.542246 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.543354 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.543362 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:37.543668 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:37.544391 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:37.544417 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:37.547762 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:37.551677 (+ 3915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:37.551687 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:37.551708 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:37.552411 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:37.552862 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:37.552870 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:37.552874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:37.553148 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:37.553923 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:37.553930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:37.554500 (+ 570us) client_negotiation.cc:769] Sending connection context
0925 11:00:37.554832 (+ 332us) client_negotiation.cc:240] Negotiation successful
0925 11:00:37.555460 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":387,"threads_started":1}
I20260925 11:00:37.558353 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:37.532540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:55123 (local address 127.24.159.254:41643)
0925 11:00:37.532718 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:37.532725 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:37.534247 (+ 1522us) server_negotiation.cc:407] Connection header received
0925 11:00:37.536445 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:37.536451 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:37.536538 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:37.536654 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:37.538703 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.539808 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.542371 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.542905 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.543040 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:37.548075 (+ 5035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:37.548082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:37.548087 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:37.548130 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:37.551499 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:37.552574 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:37.552579 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:37.552584 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:37.552729 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:37.553308 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:37.553313 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:37.553316 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:37.553698 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:37.553845 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:37.557728 (+ 3883us) server_negotiation.cc:299] Negotiation successful
0925 11:00:37.557987 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:00:37.571532 27166 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:39213
I20260925 11:00:37.571596 27300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:39213 every 8 connection(s)
I20260925 11:00:37.573104 27166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:00:37.574627 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 27166
I20260925 11:00:37.574798 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:00:37.575343 25215 external_mini_cluster.cc:1428] Setting key 231abdf601897ea621e625ad03e688a9
I20260925 11:00:37.575683 27301 heartbeater.cc:347] Connected to a master server at 127.24.159.254:41643
I20260925 11:00:37.575941 27301 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:37.576896 27301 heartbeater.cc:511] Master 127.24.159.254:41643 requested a full tablet report, sending...
I20260925 11:00:37.578375 26956 ts_manager.cc:194] Registered new tserver with Master: dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213)
I20260925 11:00:37.579401 26956 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:55123
I20260925 11:00:37.580484 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:37.564731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:39969 (local address 127.24.159.254:41643)
0925 11:00:37.564926 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:37.564934 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:37.566888 (+ 1954us) server_negotiation.cc:407] Connection header received
0925 11:00:37.567104 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:37.567109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:37.567188 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:37.567282 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:37.568612 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.569754 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.571520 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.572211 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.572335 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:37.574391 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:37.574398 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:37.574402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:37.574442 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:37.577669 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:37.578461 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:37.578466 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:37.578470 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:37.578605 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:37.579011 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:37.579020 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:37.579024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:37.579401 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:37.579551 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:37.580029 (+ 478us) server_negotiation.cc:299] Negotiation successful
0925 11:00:37.580254 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:00:37.580616 27285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:37.564709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41643 (local address 127.24.159.194:39969)
0925 11:00:37.566767 (+ 2058us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:37.566789 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:37.566947 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:37.567335 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:37.567340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:37.567985 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:37.568425 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:37.568442 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.569875 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.569879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:37.571363 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:37.571382 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:37.572404 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:37.572409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:37.572536 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:37.573100 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:37.573117 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:37.574112 (+ 995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:37.577785 (+ 3673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:37.577790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:37.577794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:37.578330 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:37.578724 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:37.578729 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:37.578733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:37.578886 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:37.579580 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:37.579585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:37.579765 (+ 180us) client_negotiation.cc:769] Sending connection context
0925 11:00:37.579899 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 11:00:37.580208 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1887,"thread_start_us":1738,"threads_started":1}
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:00:37.613310 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:41643
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:40557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:37.921175 27305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:37.921435 27305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:37.921559 27305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:37.928640 27305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:37.928771 27305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:37.928887 27305 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260925 11:00:37.928943 27305 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 11:00:37.929033 27305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:00:37.938557 27305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.159.254:41643
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:37.939792 27305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:37.941120 27305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:37.953157 27310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:37.953220 27311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:37.953189 27313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:37.954169 27305 server_base.cc:1034] running on GCE node
I20260925 11:00:37.954739 27305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:37.955555 27305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:37.956746 27305 hybrid_clock.cc:648] HybridClock initialized: now 1790334037956701 us; error 38 us; skew 500 ppm
I20260925 11:00:37.957000 27305 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:37 dist-test-slave-trms krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:37.959848 27305 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:00:37.961187 27305 webserver.cc:460] Webserver started at http://127.24.159.195:45739/ using document root <none> and password file <none>
I20260925 11:00:37.961841 27305 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:37.961937 27305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:37.962231 27305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:37.964962 27305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "869eee58115844f7b37a5c26f25598f2"
format_stamp: "Formatted at 2026-09-25 11:00:37 on dist-test-slave-trms"
server_key: "68b2954a9170e5fac99c1ffcdf73dab4"
server_key_iv: "9a242ba50e5d6e6c792a2e3a7dd5135d"
server_key_version: "encryptionkey@0"
I20260925 11:00:37.965651 27305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "869eee58115844f7b37a5c26f25598f2"
format_stamp: "Formatted at 2026-09-25 11:00:37 on dist-test-slave-trms"
server_key: "68b2954a9170e5fac99c1ffcdf73dab4"
server_key_iv: "9a242ba50e5d6e6c792a2e3a7dd5135d"
server_key_version: "encryptionkey@0"
I20260925 11:00:37.970480 27305 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 11:00:37.973666 27320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:37.975039 27305 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:37.975194 27305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "869eee58115844f7b37a5c26f25598f2"
format_stamp: "Formatted at 2026-09-25 11:00:37 on dist-test-slave-trms"
server_key: "68b2954a9170e5fac99c1ffcdf73dab4"
server_key_iv: "9a242ba50e5d6e6c792a2e3a7dd5135d"
server_key_version: "encryptionkey@0"
I20260925 11:00:37.975374 27305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:37.995409 27305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:37.999230 27305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:37.999478 27305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:38.001967 27305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:38.002065 27305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:38.002180 27305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:38.002279 27305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Sep 25 11:00:38 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334037, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:38.025215 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.002658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:36157 (local address 127.24.159.254:41643)
0925 11:00:38.002827 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.002834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.004346 (+ 1512us) server_negotiation.cc:407] Connection header received
0925 11:00:38.007091 (+ 2745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.007097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.007178 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.007279 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:38.009657 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.010550 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.012925 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.013521 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.013638 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.018283 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:38.018288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:38.018291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:38.018325 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:38.021647 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:38.022549 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:38.022554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:38.022558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:38.022734 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:38.023262 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:38.023268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:38.023271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:38.023596 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:38.023732 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.024639 (+ 907us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.024869 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:00:38.026746 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.003273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41643 (local address 127.24.159.195:36157)
0925 11:00:38.004127 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.004207 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.006752 (+ 2545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.007576 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.007591 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.008466 (+ 875us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:38.009448 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.009482 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.010709 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.010717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.012712 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.012730 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.013769 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.013777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.013994 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.014809 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:38.014850 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:38.018021 (+ 3171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:38.021784 (+ 3763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:38.021797 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:38.021814 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:38.022419 (+ 605us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:38.022873 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:38.022879 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:38.022883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:38.023098 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:38.023723 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:38.023731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:38.024224 (+ 493us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.024533 (+ 309us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.025081 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":329,"threads_started":1}
I20260925 11:00:38.057562 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.044191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:52321 (local address 127.24.159.254:41643)
0925 11:00:38.044390 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.044397 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.044867 (+ 470us) server_negotiation.cc:407] Connection header received
0925 11:00:38.045018 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.045025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.045113 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.045233 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:38.046178 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.047303 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.048594 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.049317 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.049447 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.051495 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:38.051499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:38.051502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:38.051533 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:38.054362 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:38.055242 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:38.055247 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:38.055250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:38.055412 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:38.055853 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:38.055858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:38.055862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:38.056226 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:38.056347 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.057002 (+ 655us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.057199 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 11:00:38.057643 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.044123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41643 (local address 127.24.159.195:52321)
0925 11:00:38.044777 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.044795 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.044915 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.045263 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.045266 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.045697 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:38.046021 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.046032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.047461 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.047465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.048468 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.048480 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.049482 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.049486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.049590 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.050244 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:38.050266 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:38.051258 (+ 992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:38.054491 (+ 3233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:38.054500 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:38.054504 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:38.055084 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:38.055539 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:38.055543 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:38.055546 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:38.055707 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:38.056436 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:38.056439 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:38.056692 (+ 253us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.056895 (+ 203us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.057226 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":343,"threads_started":1}
I20260925 11:00:38.060825 27305 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:39313
I20260925 11:00:38.060866 27439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:39313 every 8 connection(s)
I20260925 11:00:38.062011 27305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:00:38.063843 27440 heartbeater.cc:347] Connected to a master server at 127.24.159.254:41643
I20260925 11:00:38.064035 27440 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:38.064587 27440 heartbeater.cc:511] Master 127.24.159.254:41643 requested a full tablet report, sending...
I20260925 11:00:38.065766 26956 ts_manager.cc:194] Registered new tserver with Master: 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:38.066488 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 27305
I20260925 11:00:38.066650 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:00:38.066756 26956 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:36157
I20260925 11:00:38.067165 25215 external_mini_cluster.cc:1428] Setting key 4298bf60bb5acfd0e3b635d6f559f09e
I20260925 11:00:38.069872 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 11:00:38 dist-test-slave-trms krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 11:00:38 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:38.113579 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.099686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60962 (local address 127.24.159.254:41643)
0925 11:00:38.099895 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.099902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.100243 (+ 341us) server_negotiation.cc:407] Connection header received
0925 11:00:38.100437 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.100443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.100512 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.100597 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:38.101993 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.102780 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.104044 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.104503 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.104612 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.108684 (+ 4072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:38.108689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:38.108692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:38.108731 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:38.111227 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:38.112010 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:38.112014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:38.112016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:38.112142 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:38.112465 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:38.112468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:38.112470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:38.112723 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:38.112893 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.113126 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.113287 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:00:38.117511 27162 heartbeater.cc:503] Master 127.24.159.254:41643 was elected leader, sending a full tablet report...
I20260925 11:00:38.117625 26956 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:00:38.124595 26956 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:00:38.159554 27455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.138312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:33939 (local address 127.0.0.1:44922)
0925 11:00:38.149954 (+ 11642us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.149973 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.150099 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.151772 (+ 1673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.151780 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.151809 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:38.152137 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.152149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.154592 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.154598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.157016 (+ 2418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.157031 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.158548 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.158555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.158759 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.158819 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.158923 (+ 104us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.159098 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11518,"mutex_wait_us":1,"thread_start_us":7600,"threads_started":1}
I20260925 11:00:38.160065 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.136390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:39313 (local address 127.0.0.1:36456)
0925 11:00:38.146882 (+ 10492us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.146931 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.147152 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.152855 (+ 5703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.152861 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.152889 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:38.153330 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.153341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.155252 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.155258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.157260 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.157272 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.159308 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.159314 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.159476 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.159511 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.159597 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.159733 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9672,"thread_start_us":11004,"threads_started":1}
I20260925 11:00:38.160288 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.138833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44922 (local address 127.24.159.193:33939)
0925 11:00:38.150989 (+ 12156us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.150998 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.151027 (+ 29us) server_negotiation.cc:407] Connection header received
0925 11:00:38.151121 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.151130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.151425 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.151740 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:38.153036 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.154417 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.157194 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.158328 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.159554 (+ 1226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.159729 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.159841 (+ 112us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.160006 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11995,"thread_start_us":11662,"threads_started":1}
I20260925 11:00:38.161384 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.136541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36456 (local address 127.24.159.195:39313)
0925 11:00:38.152410 (+ 15869us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.152416 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.152441 (+ 25us) server_negotiation.cc:407] Connection header received
0925 11:00:38.152508 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.152513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.152681 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.152819 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:38.153550 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.155093 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.157430 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.158810 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.158997 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.159156 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.160323 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.161075 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15704,"thread_start_us":15521,"threads_started":1}
I20260925 11:00:38.164001 27092 tablet_service.cc:1467] Processing CreateTablet for tablet aee3b42e51be437a89f35e9993f8dac7 (DEFAULT_TABLE table=test-table [id=e3a42602e2c94d3285902e632c509c5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:38.164901 27453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.136541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39213 (local address 127.0.0.1:38404)
0925 11:00:38.147691 (+ 11150us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.147709 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.147827 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.158133 (+ 10306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.158137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.158163 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:38.158582 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.158596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.160298 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.160303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.162026 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.162040 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.164011 (+ 1971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.164018 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.164187 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.164218 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.164305 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.164439 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11002,"thread_start_us":13667,"threads_started":1}
I20260925 11:00:38.165052 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.136835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38404 (local address 127.24.159.194:39213)
0925 11:00:38.157543 (+ 20708us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.157551 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.157578 (+ 27us) server_negotiation.cc:407] Connection header received
0925 11:00:38.157672 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.157680 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.157933 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.158117 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:38.158794 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.160140 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.162254 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.163521 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.163681 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.163843 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.164607 (+ 764us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.164782 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20547,"thread_start_us":20421,"threads_started":1}
I20260925 11:00:38.164633 27370 tablet_service.cc:1467] Processing CreateTablet for tablet aee3b42e51be437a89f35e9993f8dac7 (DEFAULT_TABLE table=test-table [id=e3a42602e2c94d3285902e632c509c5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:38.165612 27092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aee3b42e51be437a89f35e9993f8dac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:38.166041 27370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aee3b42e51be437a89f35e9993f8dac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:38.168576 27231 tablet_service.cc:1467] Processing CreateTablet for tablet aee3b42e51be437a89f35e9993f8dac7 (DEFAULT_TABLE table=test-table [id=e3a42602e2c94d3285902e632c509c5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:38.169914 27231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aee3b42e51be437a89f35e9993f8dac7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:38.177778 27459 tablet_bootstrap.cc:492] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: Bootstrap starting.
I20260925 11:00:38.179589 27460 tablet_bootstrap.cc:492] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: Bootstrap starting.
I20260925 11:00:38.181581 27459 tablet_bootstrap.cc:654] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:38.182844 27461 tablet_bootstrap.cc:492] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: Bootstrap starting.
I20260925 11:00:38.182927 27460 tablet_bootstrap.cc:654] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:38.182986 27459 log.cc:824] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:38.184229 27460 log.cc:824] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:38.185912 27461 tablet_bootstrap.cc:654] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:38.187106 27461 log.cc:824] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:38.187256 27460 tablet_bootstrap.cc:492] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: No bootstrap required, opened a new log
I20260925 11:00:38.187453 27459 tablet_bootstrap.cc:492] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: No bootstrap required, opened a new log
I20260925 11:00:38.187538 27460 ts_tablet_manager.cc:1397] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260925 11:00:38.187737 27459 ts_tablet_manager.cc:1397] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260925 11:00:38.190016 27461 tablet_bootstrap.cc:492] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: No bootstrap required, opened a new log
I20260925 11:00:38.190287 27461 ts_tablet_manager.cc:1397] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260925 11:00:38.195304 27460 raft_consensus.cc:348] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } }
I20260925 11:00:38.195654 27460 raft_consensus.cc:374] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:38.195742 27460 raft_consensus.cc:729] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13eb3805718c4b1c81d183e40ae3c8f1, State: Initialized, Role: FOLLOWER
I20260925 11:00:38.196357 27460 consensus_queue.cc:260] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } }
I20260925 11:00:38.197804 27459 raft_consensus.cc:348] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } }
I20260925 11:00:38.198235 27459 raft_consensus.cc:374] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:38.198338 27459 raft_consensus.cc:729] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869eee58115844f7b37a5c26f25598f2, State: Initialized, Role: FOLLOWER
I20260925 11:00:38.198979 27459 consensus_queue.cc:260] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } }
I20260925 11:00:38.199748 27460 ts_tablet_manager.cc:1428] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260925 11:00:38.200294 27461 raft_consensus.cc:348] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } }
I20260925 11:00:38.200703 27461 raft_consensus.cc:374] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:38.200794 27461 raft_consensus.cc:729] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd670fbd634028b7f3b5f14401f160, State: Initialized, Role: FOLLOWER
I20260925 11:00:38.200922 27440 heartbeater.cc:503] Master 127.24.159.254:41643 was elected leader, sending a full tablet report...
I20260925 11:00:38.201508 27459 ts_tablet_manager.cc:1428] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260925 11:00:38.201326 27461 consensus_queue.cc:260] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } }
I20260925 11:00:38.202907 27301 heartbeater.cc:503] Master 127.24.159.254:41643 was elected leader, sending a full tablet report...
I20260925 11:00:38.203454 27461 ts_tablet_manager.cc:1428] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
W20260925 11:00:38.313479 27441 tablet.cc:2369] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:00:38.325322 27302 tablet.cc:2369] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:00:38.360749 27163 tablet.cc:2369] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:00:38.423638 27466 raft_consensus.cc:482] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:38.423949 27466 raft_consensus.cc:504] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } }
I20260925 11:00:38.425658 27466 leader_election.cc:288] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213), 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939)
I20260925 11:00:38.431088 27329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.426093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39213 (local address 127.24.159.195:60059)
0925 11:00:38.426291 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.426314 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.426446 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.426817 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.426822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.426849 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:38.427019 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.427027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.428011 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.428016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.429311 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.429326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.430389 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.430394 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.430561 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.430584 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.430664 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.430790 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":1}
I20260925 11:00:38.431085 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.425997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:60059 (local address 127.24.159.194:39213)
0925 11:00:38.426247 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.426254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.426387 (+ 133us) server_negotiation.cc:407] Connection header received
0925 11:00:38.426637 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.426641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.426701 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.426774 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:38.427153 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.427876 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.429473 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.430254 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.430392 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.430465 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.430702 (+ 237us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.430822 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 11:00:38.431710 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.426359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:49321 (local address 127.24.159.193:33939)
0925 11:00:38.426578 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.426583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.427003 (+ 420us) server_negotiation.cc:407] Connection header received
0925 11:00:38.427171 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.427175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.427230 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.427302 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:38.427770 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.428439 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.430062 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.430851 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.431007 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.431075 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.431369 (+ 294us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.431497 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:00:38.431746 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.426106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:33939 (local address 127.24.159.195:49321)
0925 11:00:38.426895 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.426910 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.427044 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.427326 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.427331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.427352 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:38.427630 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.427639 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.428567 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.428572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.429912 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.429925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.431051 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.431058 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.431216 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.431252 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.431333 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.431454 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":123,"thread_start_us":373,"threads_started":1}
I20260925 11:00:38.432181 27251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aee3b42e51be437a89f35e9993f8dac7" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160" is_pre_election: true
I20260925 11:00:38.432646 27251 raft_consensus.cc:2454] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 0.
I20260925 11:00:38.432647 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aee3b42e51be437a89f35e9993f8dac7" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" is_pre_election: true
I20260925 11:00:38.433099 27112 raft_consensus.cc:2454] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 0.
I20260925 11:00:38.433336 27324 leader_election.cc:302] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 869eee58115844f7b37a5c26f25598f2, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:38.433729 27466 raft_consensus.cc:2781] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:38.433872 27466 raft_consensus.cc:482] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:38.433959 27466 raft_consensus.cc:3037] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:38.436065 27466 raft_consensus.cc:504] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } }
I20260925 11:00:38.436691 27466 leader_election.cc:288] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 election: Requested vote from peers dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213), 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939)
I20260925 11:00:38.437165 27251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aee3b42e51be437a89f35e9993f8dac7" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160"
I20260925 11:00:38.437165 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aee3b42e51be437a89f35e9993f8dac7" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1"
I20260925 11:00:38.437379 27251 raft_consensus.cc:3037] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:38.437361 27112 raft_consensus.cc:3037] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:38.440434 27112 raft_consensus.cc:2454] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 1.
I20260925 11:00:38.440424 27251 raft_consensus.cc:2454] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 1.
I20260925 11:00:38.440802 27321 leader_election.cc:302] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1, 869eee58115844f7b37a5c26f25598f2; no voters:
I20260925 11:00:38.441123 27466 raft_consensus.cc:2781] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:38.441524 27466 raft_consensus.cc:686] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 1 LEADER]: Becoming Leader. State: Replica: 869eee58115844f7b37a5c26f25598f2, State: Running, Role: LEADER
I20260925 11:00:38.442009 27466 consensus_queue.cc:237] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } }
I20260925 11:00:38.446424 26955 catalog_manager.cc:5495] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 869eee58115844f7b37a5c26f25598f2 (127.24.159.195). New cstate: current_term: 1 leader_uuid: "869eee58115844f7b37a5c26f25598f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } health_report { overall_health: UNKNOWN } } }
Sep 25 11:00:38 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334036, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:38.557427 27004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.540452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.24.159.254:41643)
0925 11:00:38.540610 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.540614 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.541149 (+ 535us) server_negotiation.cc:407] Connection header received
0925 11:00:38.541313 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.541317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.541384 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.541473 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:38.542746 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.543517 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.544787 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.545210 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.545312 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.552331 (+ 7019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:38.552335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:38.552338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:38.552366 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:38.554871 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:38.555521 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:38.555526 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:38.555529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:38.555650 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:38.555993 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:38.555998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:38.556001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:38.556246 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:38.556368 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.556775 (+ 407us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.557077 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":50}
I20260925 11:00:38.557425 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.540501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41643 (local address 127.0.0.1:60966)
0925 11:00:38.541056 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.541072 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.541209 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.541533 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.541538 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.542322 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:38.542612 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.542622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.543634 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.543639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.544650 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.544661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.545358 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.545364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.545526 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.549685 (+ 4159us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:38.549707 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:38.552147 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:38.555006 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:38.555019 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:38.555028 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:38.555404 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:38.555752 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:38.555757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:38.555761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:38.555901 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:38.556363 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:38.556370 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:38.556540 (+ 170us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.556672 (+ 132us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.556942 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":286,"threads_started":1}
I20260925 11:00:38.562254 26955 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:60966:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 11:00:38.563266 26955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:00:38.569530 27231 tablet_service.cc:1467] Processing CreateTablet for tablet 6740266a7567422380c862a21baf4726 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81156d178a894e6b9a2ec79f707212ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 11:00:38.569486 27092 tablet_service.cc:1467] Processing CreateTablet for tablet 6740266a7567422380c862a21baf4726 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81156d178a894e6b9a2ec79f707212ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 11:00:38.569914 27370 tablet_service.cc:1467] Processing CreateTablet for tablet 6740266a7567422380c862a21baf4726 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=81156d178a894e6b9a2ec79f707212ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 11:00:38.570523 27092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6740266a7567422380c862a21baf4726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:38.570523 27231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6740266a7567422380c862a21baf4726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:38.570734 27370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6740266a7567422380c862a21baf4726. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:38.577366 27459 tablet_bootstrap.cc:492] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: Bootstrap starting.
I20260925 11:00:38.577569 27460 tablet_bootstrap.cc:492] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: Bootstrap starting.
I20260925 11:00:38.578938 27460 tablet_bootstrap.cc:654] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:38.578986 27459 tablet_bootstrap.cc:654] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:38.580061 27461 tablet_bootstrap.cc:492] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: Bootstrap starting.
I20260925 11:00:38.581420 27461 tablet_bootstrap.cc:654] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:38.582794 27460 tablet_bootstrap.cc:492] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: No bootstrap required, opened a new log
I20260925 11:00:38.583014 27460 ts_tablet_manager.cc:1397] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260925 11:00:38.583649 27460 raft_consensus.cc:348] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.583892 27460 raft_consensus.cc:374] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:38.583992 27460 raft_consensus.cc:729] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13eb3805718c4b1c81d183e40ae3c8f1, State: Initialized, Role: FOLLOWER
I20260925 11:00:38.584241 27460 consensus_queue.cc:260] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.584658 27465 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.584933 27465 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:38.585843 27460 ts_tablet_manager.cc:1428] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 11:00:38.589526 27461 tablet_bootstrap.cc:492] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: No bootstrap required, opened a new log
I20260925 11:00:38.589687 27461 ts_tablet_manager.cc:1397] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.001s
I20260925 11:00:38.590512 27461 raft_consensus.cc:348] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.590780 27461 raft_consensus.cc:374] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:38.590884 27461 raft_consensus.cc:729] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd670fbd634028b7f3b5f14401f160, State: Initialized, Role: FOLLOWER
I20260925 11:00:38.590929 27459 tablet_bootstrap.cc:492] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: No bootstrap required, opened a new log
I20260925 11:00:38.591061 27459 ts_tablet_manager.cc:1397] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.011s
I20260925 11:00:38.591120 27461 consensus_queue.cc:260] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.591765 27461 ts_tablet_manager.cc:1428] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 11:00:38.591593 27467 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.591768 27459 raft_consensus.cc:348] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.591979 27459 raft_consensus.cc:374] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:38.592002 27467 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:38.592062 27459 raft_consensus.cc:729] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869eee58115844f7b37a5c26f25598f2, State: Initialized, Role: FOLLOWER
I20260925 11:00:38.592288 27459 consensus_queue.cc:260] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.592723 27459 ts_tablet_manager.cc:1428] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 11:00:38.592666 27466 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.592950 27466 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:38.617697 27465 raft_consensus.cc:482] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:38.617936 27465 raft_consensus.cc:504] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.619542 27465 leader_election.cc:288] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:38.625929 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.620355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:53707 (local address 127.24.159.195:39313)
0925 11:00:38.620529 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.620535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.620988 (+ 453us) server_negotiation.cc:407] Connection header received
0925 11:00:38.621131 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.621136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.621192 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.621261 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:38.621841 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.622590 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.624265 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.625025 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.625164 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.625237 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.625629 (+ 392us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.625738 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260925 11:00:38.625976 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.620031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:41581 (local address 127.24.159.194:39213)
0925 11:00:38.620223 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.620230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.620798 (+ 568us) server_negotiation.cc:407] Connection header received
0925 11:00:38.620976 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.620980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.621041 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.621111 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:38.621676 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.622468 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.624258 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.625029 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.625165 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.625258 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.625595 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.625714 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:00:38.626045 27480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.619753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39213 (local address 127.24.159.193:41581)
0925 11:00:38.620690 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.620709 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.620842 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.621137 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.621142 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.621166 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:38.621525 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.621534 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.622584 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.622589 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.624116 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.624128 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.625281 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.625287 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.625431 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.625479 (+ 48us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.625559 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.625677 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":821,"mutex_wait_us":75,"thread_start_us":650,"threads_started":1}
I20260925 11:00:38.626600 27481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.619997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:39313 (local address 127.24.159.193:53707)
0925 11:00:38.620876 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.620890 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.621015 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.621299 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.621303 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.621331 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:38.621677 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.621687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.622721 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.622726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.624118 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.624130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.625231 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.625237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.625426 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.625482 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.626154 (+ 672us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.626290 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":9,"thread_start_us":567,"threads_started":1}
I20260925 11:00:38.626698 27251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6740266a7567422380c862a21baf4726" candidate_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160" is_pre_election: true
I20260925 11:00:38.626933 27251 raft_consensus.cc:2454] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13eb3805718c4b1c81d183e40ae3c8f1 in term 0.
I20260925 11:00:38.627347 27046 leader_election.cc:302] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:38.627389 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6740266a7567422380c862a21baf4726" candidate_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2" is_pre_election: true
I20260925 11:00:38.627698 27465 raft_consensus.cc:2781] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:38.627717 27390 raft_consensus.cc:2454] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13eb3805718c4b1c81d183e40ae3c8f1 in term 0.
I20260925 11:00:38.627823 27465 raft_consensus.cc:482] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:38.627925 27465 raft_consensus.cc:3037] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:38.629946 27465 raft_consensus.cc:504] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.630473 27465 leader_election.cc:288] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [CANDIDATE]: Term 1 election: Requested vote from peers dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:38.630828 27251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6740266a7567422380c862a21baf4726" candidate_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160"
I20260925 11:00:38.630903 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6740266a7567422380c862a21baf4726" candidate_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2"
I20260925 11:00:38.631040 27251 raft_consensus.cc:3037] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:38.631089 27390 raft_consensus.cc:3037] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:38.633836 27251 raft_consensus.cc:2454] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13eb3805718c4b1c81d183e40ae3c8f1 in term 1.
I20260925 11:00:38.633850 27390 raft_consensus.cc:2454] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13eb3805718c4b1c81d183e40ae3c8f1 in term 1.
I20260925 11:00:38.634227 27046 leader_election.cc:302] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1, 869eee58115844f7b37a5c26f25598f2, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:38.634524 27465 raft_consensus.cc:2781] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:38.634927 27465 raft_consensus.cc:686] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 LEADER]: Becoming Leader. State: Replica: 13eb3805718c4b1c81d183e40ae3c8f1, State: Running, Role: LEADER
I20260925 11:00:38.635403 27465 consensus_queue.cc:237] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.638504 27482 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: TxnStatusTablet state changed. Reason: New leader 13eb3805718c4b1c81d183e40ae3c8f1. Latest consensus state: current_term: 1 leader_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.638895 27482 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: This TxnStatusTablet replica's current role is: LEADER
I20260925 11:00:38.639539 27484 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 11:00:38.639835 27484 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 11:00:38.639812 26955 catalog_manager.cc:5495] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193). New cstate: current_term: 1 leader_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } health_report { overall_health: UNKNOWN } } }
I20260925 11:00:38.656989 27476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.651153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:33939 (local address 127.0.0.1:44928)
0925 11:00:38.651400 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.651416 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.651527 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.651994 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.651997 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.652007 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:00:38.652148 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.652155 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.653565 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.653568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.654574 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.654586 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.655461 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.655465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.655588 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.655638 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:00:38.656580 (+ 942us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:00:38.656593 (+ 13us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.656644 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.656754 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260925 11:00:38.657104 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.651148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.24.159.193:33939)
0925 11:00:38.651324 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.651332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.651482 (+ 150us) server_negotiation.cc:407] Connection header received
0925 11:00:38.651694 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.651700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.651804 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.651978 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:38.652268 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.653461 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.654700 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.655327 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.655463 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.655815 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:38.656484 (+ 669us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:38.656621 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.656672 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.656751 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 11:00:38.665282 27251 raft_consensus.cc:1261] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Refusing update from remote peer 13eb3805718c4b1c81d183e40ae3c8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:38.665266 27390 raft_consensus.cc:1261] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Refusing update from remote peer 13eb3805718c4b1c81d183e40ae3c8f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:38.666235 27465 consensus_queue.cc:1035] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:38.666519 27482 consensus_queue.cc:1035] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:38.681259 27466 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: TxnStatusTablet state changed. Reason: New leader 13eb3805718c4b1c81d183e40ae3c8f1. Latest consensus state: current_term: 1 leader_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.681581 27466 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:38.683948 27487 mvcc.cc:205] Tried to move back new op lower bound from 7333208222362845184 to 7333208222254268416. Current Snapshot: MvccSnapshot[applied={T|T < 7333208222362845184}]
I20260925 11:00:38.686546 27467 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: TxnStatusTablet state changed. Reason: New leader 13eb3805718c4b1c81d183e40ae3c8f1. Latest consensus state: current_term: 1 leader_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.686823 27467 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:38.688203 27465 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.688511 27465 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: This TxnStatusTablet replica's current role is: LEADER
I20260925 11:00:38.690527 27489 mvcc.cc:205] Tried to move back new op lower bound from 7333208222362845184 to 7333208222254268416. Current Snapshot: MvccSnapshot[applied={T|T < 7333208222362845184}]
I20260925 11:00:38.691300 27467 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.691536 27467 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P dafd670fbd634028b7f3b5f14401f160: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:38.694054 27466 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.694545 27466 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P 869eee58115844f7b37a5c26f25598f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:00:38.697209 27482 tablet_replica.cc:439] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } } }
I20260925 11:00:38.697460 27482 tablet_replica.cc:442] T 6740266a7567422380c862a21baf4726 P 13eb3805718c4b1c81d183e40ae3c8f1: This TxnStatusTablet replica's current role is: LEADER
I20260925 11:00:38.706794 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.700497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36472 (local address 127.24.159.195:39313)
0925 11:00:38.700638 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.700643 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.701168 (+ 525us) server_negotiation.cc:407] Connection header received
0925 11:00:38.701327 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.701332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.701445 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.701595 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:38.701963 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.703136 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.704415 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.705026 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.705145 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.705484 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:38.706216 (+ 732us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:38.706358 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.706406 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.706487 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 11:00:38.721872 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.716676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:34379 (local address 127.24.159.193:33939)
0925 11:00:38.716836 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.716840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.717369 (+ 529us) server_negotiation.cc:407] Connection header received
0925 11:00:38.717561 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.717565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.717627 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.717703 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:38.718147 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.718953 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.720242 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.720645 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.720748 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.721168 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:38.721276 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:38.721345 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.721489 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.721568 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 11:00:38.721932 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.716700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:33939 (local address 127.24.159.195:34379)
0925 11:00:38.717274 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.717289 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.717428 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.717727 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.717731 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.717742 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:00:38.718005 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.718013 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.719074 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.719080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.720110 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.720122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.720807 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.720813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.720939 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.720990 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:00:38.721372 (+ 382us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:00:38.721390 (+ 18us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.721454 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.721581 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":287,"threads_started":1}
I20260925 11:00:38.730625 27251 raft_consensus.cc:1261] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Refusing update from remote peer 869eee58115844f7b37a5c26f25598f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:38.730784 27112 raft_consensus.cc:1261] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Refusing update from remote peer 869eee58115844f7b37a5c26f25598f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:38.731346 27469 consensus_queue.cc:1035] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:38.731724 27466 consensus_queue.cc:1035] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:38.765691 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.759329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:49749 (local address 127.24.159.195:39313)
0925 11:00:38.759485 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:38.759489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:38.760363 (+ 874us) server_negotiation.cc:407] Connection header received
0925 11:00:38.760560 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:38.760565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:38.760647 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:38.760739 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:38.761347 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.762233 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.763772 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.764254 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.764339 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.764976 (+ 637us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:38.765078 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:38.765152 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:38.765322 (+ 170us) server_negotiation.cc:299] Negotiation successful
0925 11:00:38.765409 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 11:00:38.765815 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:38.759466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:39313 (local address 127.24.159.193:49749)
0925 11:00:38.760188 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:38.760280 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:38.760423 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:38.760766 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:38.760771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:38.760783 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:00:38.761186 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.761196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.762368 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.762374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.763617 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:38.763628 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:38.764543 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:38.764549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:38.764737 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:38.764811 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:00:38.765201 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:00:38.765212 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 11:00:38.765288 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 11:00:38.765425 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":387,"threads_started":1}
I20260925 11:00:38.814534 26956 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:38.814813 26956 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:38.818900 26956 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043: Sending DeleteTablet for 3 replicas of tablet aee3b42e51be437a89f35e9993f8dac7
I20260925 11:00:38.819903 27370 tablet_service.cc:1514] Processing DeleteTablet for tablet aee3b42e51be437a89f35e9993f8dac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:38 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:36456
I20260925 11:00:38.820034 27231 tablet_service.cc:1514] Processing DeleteTablet for tablet aee3b42e51be437a89f35e9993f8dac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:38 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38404
I20260925 11:00:38.821326 27092 tablet_service.cc:1514] Processing DeleteTablet for tablet aee3b42e51be437a89f35e9993f8dac7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:38 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:44922
I20260925 11:00:38.823560 26956 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:00:38.824331 26956 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:00:38.832612 27091 tablet_service.cc:1467] Processing CreateTablet for tablet 2d55e7c743e248c7a2687ade7b97d391 (DEFAULT_TABLE table=test-table [id=310fabf3f4b04f209b1c90f6b6c56646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:38.833235 27091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d55e7c743e248c7a2687ade7b97d391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:38.834348 27509 tablet_replica.cc:330] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: stopping tablet replica
I20260925 11:00:38.835412 27509 raft_consensus.cc:2229] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:38.836295 27230 tablet_service.cc:1467] Processing CreateTablet for tablet 2d55e7c743e248c7a2687ade7b97d391 (DEFAULT_TABLE table=test-table [id=310fabf3f4b04f209b1c90f6b6c56646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:38.836820 27230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d55e7c743e248c7a2687ade7b97d391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:38.837061 27369 tablet_service.cc:1467] Processing CreateTablet for tablet 2d55e7c743e248c7a2687ade7b97d391 (DEFAULT_TABLE table=test-table [id=310fabf3f4b04f209b1c90f6b6c56646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:38.837617 27369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d55e7c743e248c7a2687ade7b97d391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:38.840772 27509 raft_consensus.cc:2258] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:38.842604 27508 tablet_replica.cc:330] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: stopping tablet replica
I20260925 11:00:38.843657 27508 raft_consensus.cc:2229] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:00:38.843886 27461 tablet_bootstrap.cc:492] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: Bootstrap starting.
I20260925 11:00:38.844228 27459 tablet_bootstrap.cc:492] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: Bootstrap starting.
I20260925 11:00:38.844329 27508 raft_consensus.cc:2258] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:38.845022 27510 tablet_replica.cc:330] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: stopping tablet replica
I20260925 11:00:38.845479 27461 tablet_bootstrap.cc:654] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:38.845719 27459 tablet_bootstrap.cc:654] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:38.845404 27510 raft_consensus.cc:2229] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:38.847558 27510 raft_consensus.cc:2258] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:38.848596 27460 tablet_bootstrap.cc:492] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: Bootstrap starting.
I20260925 11:00:38.849608 27510 ts_tablet_manager.cc:1905] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:38.850183 27460 tablet_bootstrap.cc:654] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:38.851203 27508 ts_tablet_manager.cc:1905] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:38.855204 27509 ts_tablet_manager.cc:1905] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:38.862324 27460 tablet_bootstrap.cc:492] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: No bootstrap required, opened a new log
I20260925 11:00:38.862498 27460 ts_tablet_manager.cc:1397] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.007s
I20260925 11:00:38.863197 27460 raft_consensus.cc:348] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.863438 27460 raft_consensus.cc:374] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:38.863510 27460 raft_consensus.cc:729] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13eb3805718c4b1c81d183e40ae3c8f1, State: Initialized, Role: FOLLOWER
I20260925 11:00:38.863768 27460 consensus_queue.cc:260] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.864823 27461 tablet_bootstrap.cc:492] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: No bootstrap required, opened a new log
I20260925 11:00:38.865024 27461 ts_tablet_manager.cc:1397] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.004s
I20260925 11:00:38.865764 27461 raft_consensus.cc:348] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.866029 27461 raft_consensus.cc:374] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:38.866137 27461 raft_consensus.cc:729] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd670fbd634028b7f3b5f14401f160, State: Initialized, Role: FOLLOWER
I20260925 11:00:38.866616 27460 ts_tablet_manager.cc:1428] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260925 11:00:38.866501 27461 consensus_queue.cc:260] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.869199 27510 ts_tablet_manager.cc:1918] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:00:38.870262 27461 ts_tablet_manager.cc:1428] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 11:00:38.870304 27510 log.cc:1196] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/aee3b42e51be437a89f35e9993f8dac7
I20260925 11:00:38.870893 27510 ts_tablet_manager.cc:1939] T aee3b42e51be437a89f35e9993f8dac7 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting consensus metadata
I20260925 11:00:38.872694 27508 ts_tablet_manager.cc:1918] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:00:38.872920 27508 log.cc:1196] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/aee3b42e51be437a89f35e9993f8dac7
I20260925 11:00:38.873477 27508 ts_tablet_manager.cc:1939] T aee3b42e51be437a89f35e9993f8dac7 P 869eee58115844f7b37a5c26f25598f2: Deleting consensus metadata
I20260925 11:00:38.874629 26940 catalog_manager.cc:4841] TS 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939): tablet aee3b42e51be437a89f35e9993f8dac7 (table test-table [id=e3a42602e2c94d3285902e632c509c5b]) successfully deleted
I20260925 11:00:38.875183 26941 catalog_manager.cc:4841] TS 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313): tablet aee3b42e51be437a89f35e9993f8dac7 (table test-table [id=e3a42602e2c94d3285902e632c509c5b]) successfully deleted
I20260925 11:00:38.876543 27459 tablet_bootstrap.cc:492] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: No bootstrap required, opened a new log
I20260925 11:00:38.876750 27459 ts_tablet_manager.cc:1397] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.010s
I20260925 11:00:38.877526 27459 raft_consensus.cc:348] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.877825 27459 raft_consensus.cc:374] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:38.877952 27459 raft_consensus.cc:729] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869eee58115844f7b37a5c26f25598f2, State: Initialized, Role: FOLLOWER
I20260925 11:00:38.878072 27509 ts_tablet_manager.cc:1918] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:00:38.878276 27509 log.cc:1196] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/aee3b42e51be437a89f35e9993f8dac7
I20260925 11:00:38.878288 27459 consensus_queue.cc:260] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:38.878782 27509 ts_tablet_manager.cc:1939] T aee3b42e51be437a89f35e9993f8dac7 P dafd670fbd634028b7f3b5f14401f160: Deleting consensus metadata
I20260925 11:00:38.879088 27459 ts_tablet_manager.cc:1428] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 11:00:38.880436 26943 catalog_manager.cc:4841] TS dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213): tablet aee3b42e51be437a89f35e9993f8dac7 (table test-table [id=e3a42602e2c94d3285902e632c509c5b]) successfully deleted
I20260925 11:00:39.142452 27467 raft_consensus.cc:482] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:39.142688 27467 raft_consensus.cc:504] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:39.144589 27467 leader_election.cc:288] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:39.149813 27465 raft_consensus.cc:482] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:39.150374 27505 raft_consensus.cc:482] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:39.150565 27505 raft_consensus.cc:504] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:39.151420 27505 leader_election.cc:288] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213)
I20260925 11:00:39.152465 27454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:39.145189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:34513 (local address 127.24.159.195:39313)
0925 11:00:39.145411 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:39.145420 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:39.145828 (+ 408us) server_negotiation.cc:407] Connection header received
0925 11:00:39.146049 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:39.146055 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:39.146138 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:39.146278 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:39.147051 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:39.148061 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:39.149895 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:39.151323 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:39.151449 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:39.151523 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:39.152029 (+ 506us) server_negotiation.cc:299] Negotiation successful
0925 11:00:39.152155 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 11:00:39.152561 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" is_pre_election: true
I20260925 11:00:39.152972 27250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160" is_pre_election: true
I20260925 11:00:39.153128 27515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:39.144925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:39313 (local address 127.24.159.194:34513)
0925 11:00:39.145727 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:39.145740 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:39.145859 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:39.146468 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:39.146472 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:39.146501 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:39.146869 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:39.146882 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:39.148183 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:39.148186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:39.149698 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:39.149713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:39.151662 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:39.151667 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:39.151835 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:39.151869 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 11:00:39.152623 (+ 754us) client_negotiation.cc:240] Negotiation successful
0925 11:00:39.152754 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":599,"threads_started":1}
I20260925 11:00:39.153255 27250 raft_consensus.cc:2454] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 0.
I20260925 11:00:39.153673 27324 leader_election.cc:302] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 869eee58115844f7b37a5c26f25598f2, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:39.153961 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2" is_pre_election: true
I20260925 11:00:39.154165 27390 raft_consensus.cc:2454] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafd670fbd634028b7f3b5f14401f160 in term 0.
I20260925 11:00:39.150015 27465 raft_consensus.cc:504] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:39.154476 27458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:39.144982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:53767 (local address 127.24.159.193:33939)
0925 11:00:39.145159 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:39.145165 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:39.145631 (+ 466us) server_negotiation.cc:407] Connection header received
0925 11:00:39.145968 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:39.145974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:39.146046 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:39.146121 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:00:39.146696 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:39.147624 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:39.150709 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:39.151917 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:39.152068 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:39.152144 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:39.154059 (+ 1915us) server_negotiation.cc:299] Negotiation successful
0925 11:00:39.154217 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:00:39.154901 27514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:39.144718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:33939 (local address 127.24.159.194:53767)
0925 11:00:39.145514 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:39.145535 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:39.145750 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:39.146147 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:39.146151 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:39.146207 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:00:39.146537 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:39.146554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:39.147778 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:39.147784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:39.149900 (+ 2116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:39.149915 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:39.152904 (+ 2989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:39.152912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:39.153118 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:39.153916 (+ 798us) client_negotiation.cc:769] Sending connection context
0925 11:00:39.154016 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 11:00:39.154255 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":1,"thread_start_us":517,"threads_started":1}
I20260925 11:00:39.154892 27465 leader_election.cc:288] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:39.155514 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2" is_pre_election: true
I20260925 11:00:39.155615 27111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" is_pre_election: true
I20260925 11:00:39.155714 27390 raft_consensus.cc:2454] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13eb3805718c4b1c81d183e40ae3c8f1 in term 0.
I20260925 11:00:39.155673 27183 leader_election.cc:302] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 869eee58115844f7b37a5c26f25598f2, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:39.156042 27467 raft_consensus.cc:2781] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:39.156064 27112 raft_consensus.cc:2454] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 0.
I20260925 11:00:39.156155 27467 raft_consensus.cc:482] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:39.156236 27467 raft_consensus.cc:3037] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:39.156879 27044 leader_election.cc:302] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1, 869eee58115844f7b37a5c26f25598f2; no voters:
I20260925 11:00:39.157135 27465 raft_consensus.cc:2781] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:39.157229 27465 raft_consensus.cc:482] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:39.157298 27465 raft_consensus.cc:3037] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:39.157361 27250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160" is_pre_election: true
I20260925 11:00:39.158022 27505 raft_consensus.cc:2781] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:39.158142 27505 raft_consensus.cc:482] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:39.158262 27505 raft_consensus.cc:3037] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:39.159325 27467 raft_consensus.cc:504] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:39.160054 27465 raft_consensus.cc:504] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:39.160362 27250 raft_consensus.cc:2379] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 13eb3805718c4b1c81d183e40ae3c8f1 in current term 1: Already voted for candidate dafd670fbd634028b7f3b5f14401f160 in this term.
I20260925 11:00:39.161041 27467 leader_election.cc:288] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 1 election: Requested vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:39.161305 27505 raft_consensus.cc:504] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:39.161737 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1"
I20260925 11:00:39.161756 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2"
I20260925 11:00:39.161932 27505 leader_election.cc:288] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 election: Requested vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213)
I20260925 11:00:39.161978 27112 raft_consensus.cc:2379] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dafd670fbd634028b7f3b5f14401f160 in current term 1: Already voted for candidate 13eb3805718c4b1c81d183e40ae3c8f1 in this term.
I20260925 11:00:39.162005 27390 raft_consensus.cc:2379] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate dafd670fbd634028b7f3b5f14401f160 in current term 1: Already voted for candidate 869eee58115844f7b37a5c26f25598f2 in this term.
I20260925 11:00:39.162706 27251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160"
I20260925 11:00:39.162751 27250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160"
I20260925 11:00:39.162902 27465 leader_election.cc:288] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [CANDIDATE]: Term 1 election: Requested vote from peers dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:39.162950 27251 raft_consensus.cc:2379] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 869eee58115844f7b37a5c26f25598f2 in current term 1: Already voted for candidate dafd670fbd634028b7f3b5f14401f160 in this term.
I20260925 11:00:39.163405 27183 leader_election.cc:302] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dafd670fbd634028b7f3b5f14401f160; no voters: 13eb3805718c4b1c81d183e40ae3c8f1, 869eee58115844f7b37a5c26f25598f2
I20260925 11:00:39.163609 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1"
I20260925 11:00:39.163810 27112 raft_consensus.cc:2379] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 869eee58115844f7b37a5c26f25598f2 in current term 1: Already voted for candidate 13eb3805718c4b1c81d183e40ae3c8f1 in this term.
I20260925 11:00:39.163780 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2"
I20260925 11:00:39.163937 27467 raft_consensus.cc:2726] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 11:00:39.164004 27390 raft_consensus.cc:2379] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 13eb3805718c4b1c81d183e40ae3c8f1 in current term 1: Already voted for candidate 869eee58115844f7b37a5c26f25598f2 in this term.
I20260925 11:00:39.164287 27321 leader_election.cc:302] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 869eee58115844f7b37a5c26f25598f2; no voters: 13eb3805718c4b1c81d183e40ae3c8f1, dafd670fbd634028b7f3b5f14401f160
I20260925 11:00:39.164395 27044 leader_election.cc:302] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1; no voters: 869eee58115844f7b37a5c26f25598f2, dafd670fbd634028b7f3b5f14401f160
I20260925 11:00:39.164600 27505 raft_consensus.cc:2726] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 11:00:39.164705 27465 raft_consensus.cc:2726] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260925 11:00:40.826375 27516 raft_consensus.cc:482] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:40.826565 27516 raft_consensus.cc:504] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:40.827162 27516 leader_election.cc:288] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:40.827756 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" is_pre_election: true
I20260925 11:00:40.827737 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2" is_pre_election: true
I20260925 11:00:40.827987 27390 raft_consensus.cc:2454] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafd670fbd634028b7f3b5f14401f160 in term 1.
I20260925 11:00:40.827987 27112 raft_consensus.cc:2454] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafd670fbd634028b7f3b5f14401f160 in term 1.
I20260925 11:00:40.828438 27182 leader_election.cc:302] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:40.828707 27516 raft_consensus.cc:2781] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 11:00:40.828828 27516 raft_consensus.cc:482] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:40.828910 27516 raft_consensus.cc:3037] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Advancing to term 2
I20260925 11:00:40.831025 27516 raft_consensus.cc:504] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:40.831516 27516 leader_election.cc:288] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 2 election: Requested vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:40.831956 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1"
I20260925 11:00:40.831998 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d55e7c743e248c7a2687ade7b97d391" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2"
I20260925 11:00:40.832156 27112 raft_consensus.cc:3037] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Advancing to term 2
I20260925 11:00:40.832178 27390 raft_consensus.cc:3037] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Advancing to term 2
I20260925 11:00:40.834019 27112 raft_consensus.cc:2454] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dafd670fbd634028b7f3b5f14401f160 in term 2.
I20260925 11:00:40.834080 27390 raft_consensus.cc:2454] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dafd670fbd634028b7f3b5f14401f160 in term 2.
I20260925 11:00:40.834414 27182 leader_election.cc:302] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:40.834672 27516 raft_consensus.cc:2781] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 11:00:40.835016 27516 raft_consensus.cc:686] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 2 LEADER]: Becoming Leader. State: Replica: dafd670fbd634028b7f3b5f14401f160, State: Running, Role: LEADER
I20260925 11:00:40.835412 27516 consensus_queue.cc:237] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:40.838534 26955 catalog_manager.cc:5495] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 reported cstate change: term changed from 0 to 2, leader changed from <none> to dafd670fbd634028b7f3b5f14401f160 (127.24.159.194). New cstate: current_term: 2 leader_uuid: "dafd670fbd634028b7f3b5f14401f160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } health_report { overall_health: UNKNOWN } } }
I20260925 11:00:40.962322 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:40.956552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44962 (local address 127.24.159.194:39213)
0925 11:00:40.956996 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:40.957001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:40.957486 (+ 485us) server_negotiation.cc:407] Connection header received
0925 11:00:40.957642 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:40.957646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:40.957732 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:40.957866 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:40.958243 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:40.958904 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:40.960169 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:40.960706 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:40.960797 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:40.961123 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:40.961729 (+ 606us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:40.961826 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:40.961942 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 11:00:40.962006 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":223,"threads_started":1}
I20260925 11:00:40.976102 27539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:40.971219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:33939 (local address 127.24.159.194:43949)
0925 11:00:40.971918 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:40.971934 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:40.972048 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:40.972305 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:40.972308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:40.972316 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:00:40.972552 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:40.972559 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:40.973458 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:40.973462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:40.974540 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:40.974552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:40.975216 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:40.975220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:40.975327 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:40.975374 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:00:40.975665 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:00:40.975672 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 11:00:40.975724 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 11:00:40.975820 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":300,"threads_started":1}
I20260925 11:00:40.976123 27540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:40.971275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:43949 (local address 127.24.159.193:33939)
0925 11:00:40.971718 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:40.971722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:40.972010 (+ 288us) server_negotiation.cc:407] Connection header received
0925 11:00:40.972144 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:40.972147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:40.972212 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:40.972291 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:40.972669 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:40.973364 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:40.974682 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:40.975090 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:40.975184 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:40.975504 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:40.975597 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:40.975659 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:40.975750 (+ 91us) server_negotiation.cc:299] Negotiation successful
0925 11:00:40.975808 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":241,"threads_started":1}
I20260925 11:00:40.983484 27390 raft_consensus.cc:1261] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 2 FOLLOWER]: Refusing update from remote peer dafd670fbd634028b7f3b5f14401f160: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 11:00:40.983536 27112 raft_consensus.cc:1261] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 2 FOLLOWER]: Refusing update from remote peer dafd670fbd634028b7f3b5f14401f160: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260925 11:00:40.984100 27517 consensus_queue.cc:1035] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [LEADER]: Connected to new peer: Peer: permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:40.984516 27516 consensus_queue.cc:1035] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:41.008658 27536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:41.003699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:50437 (local address 127.24.159.194:39213)
0925 11:00:41.003865 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:41.003871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:41.004014 (+ 143us) server_negotiation.cc:407] Connection header received
0925 11:00:41.004193 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:41.004199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:41.004261 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:41.004326 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:41.004725 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:41.005563 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:41.007177 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:41.007588 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:41.007691 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:41.008033 (+ 342us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:41.008130 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:41.008198 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:41.008316 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 11:00:41.008387 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:00:41.008776 27545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:41.003445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39213 (local address 127.24.159.193:50437)
0925 11:00:41.003932 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:41.003949 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:41.004073 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:41.004352 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:41.004356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:41.004367 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:00:41.004597 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:41.004605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:41.005671 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:41.005677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:41.007059 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:41.007070 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:41.007728 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:41.007734 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:41.007866 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:41.007900 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:00:41.008214 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:00:41.008225 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 11:00:41.008290 (+ 65us) client_negotiation.cc:240] Negotiation successful
0925 11:00:41.008427 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":313,"threads_started":1}
W20260925 11:00:41.009735 27527 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 11:00:42.601795 26955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:42.602026 26955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:42.605922 26955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043: Sending DeleteTablet for 3 replicas of tablet 2d55e7c743e248c7a2687ade7b97d391
I20260925 11:00:42.606675 27230 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55e7c743e248c7a2687ade7b97d391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:42 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38404
I20260925 11:00:42.606678 27091 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55e7c743e248c7a2687ade7b97d391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:42 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:44922
I20260925 11:00:42.606833 27369 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d55e7c743e248c7a2687ade7b97d391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:42 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:36456
I20260925 11:00:42.617938 27573 tablet_replica.cc:330] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: stopping tablet replica
I20260925 11:00:42.618294 27573 raft_consensus.cc:2229] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 2 LEADER]: Raft consensus shutting down.
I20260925 11:00:42.619010 27573 raft_consensus.cc:2258] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:42.620566 27572 tablet_replica.cc:330] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: stopping tablet replica
I20260925 11:00:42.620926 27574 tablet_replica.cc:330] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: stopping tablet replica
I20260925 11:00:42.620880 27572 raft_consensus.cc:2229] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:42.621109 27574 raft_consensus.cc:2229] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:42.621126 27572 raft_consensus.cc:2258] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:42.621325 27574 raft_consensus.cc:2258] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:42.621852 27573 ts_tablet_manager.cc:1905] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:42.622241 27574 ts_tablet_manager.cc:1905] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:42.626531 27574 ts_tablet_manager.cc:1918] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.8
I20260925 11:00:42.626735 27574 log.cc:1196] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/2d55e7c743e248c7a2687ade7b97d391
I20260925 11:00:42.626824 27572 ts_tablet_manager.cc:1905] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:42.627228 27574 ts_tablet_manager.cc:1939] T 2d55e7c743e248c7a2687ade7b97d391 P 869eee58115844f7b37a5c26f25598f2: Deleting consensus metadata
I20260925 11:00:42.627888 27573 ts_tablet_manager.cc:1918] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.8
I20260925 11:00:42.628082 27573 log.cc:1196] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/2d55e7c743e248c7a2687ade7b97d391
I20260925 11:00:42.628324 26941 catalog_manager.cc:4841] TS 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313): tablet 2d55e7c743e248c7a2687ade7b97d391 (table test-table [id=310fabf3f4b04f209b1c90f6b6c56646]) successfully deleted
I20260925 11:00:42.628607 27573 ts_tablet_manager.cc:1939] T 2d55e7c743e248c7a2687ade7b97d391 P dafd670fbd634028b7f3b5f14401f160: Deleting consensus metadata
I20260925 11:00:42.629627 26943 catalog_manager.cc:4841] TS dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213): tablet 2d55e7c743e248c7a2687ade7b97d391 (table test-table [id=310fabf3f4b04f209b1c90f6b6c56646]) successfully deleted
I20260925 11:00:42.632019 27572 ts_tablet_manager.cc:1918] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.8
I20260925 11:00:42.632130 27572 log.cc:1196] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/2d55e7c743e248c7a2687ade7b97d391
I20260925 11:00:42.632467 27572 ts_tablet_manager.cc:1939] T 2d55e7c743e248c7a2687ade7b97d391 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting consensus metadata
I20260925 11:00:42.633273 26940 catalog_manager.cc:4841] TS 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939): tablet 2d55e7c743e248c7a2687ade7b97d391 (table test-table [id=310fabf3f4b04f209b1c90f6b6c56646]) successfully deleted
Sep 25 11:00:42 dist-test-slave-trms krb5kdc[26911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 11:00:42 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:42.660305 27582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:42.647144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33744 (local address 127.24.159.254:41643)
0925 11:00:42.647692 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:42.647698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:42.647721 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:00:42.647909 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:42.647914 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:42.647975 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:42.648059 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:42.649325 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:42.650155 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:42.651395 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:42.651790 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:42.651900 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:42.655495 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:42.655500 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:42.655503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:42.655538 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:42.657835 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:42.658459 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:42.658464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:42.658468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:42.658621 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:42.658956 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:42.658960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:42.658963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:42.659226 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:42.659469 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:42.659737 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 11:00:42.659980 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":265,"threads_started":1}
I20260925 11:00:42.663523 26955 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:00:42.664080 26955 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:00:42.670104 27091 tablet_service.cc:1467] Processing CreateTablet for tablet d54b00ac6eed4b4286db3b7da8747f15 (DEFAULT_TABLE table=test-table [id=88cb5bcf6c3c4a8fa6e974826901b79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:42.670347 27369 tablet_service.cc:1467] Processing CreateTablet for tablet d54b00ac6eed4b4286db3b7da8747f15 (DEFAULT_TABLE table=test-table [id=88cb5bcf6c3c4a8fa6e974826901b79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:42.670362 27230 tablet_service.cc:1467] Processing CreateTablet for tablet d54b00ac6eed4b4286db3b7da8747f15 (DEFAULT_TABLE table=test-table [id=88cb5bcf6c3c4a8fa6e974826901b79c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:42.670598 27091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d54b00ac6eed4b4286db3b7da8747f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:42.670758 27230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d54b00ac6eed4b4286db3b7da8747f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:42.670756 27369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d54b00ac6eed4b4286db3b7da8747f15. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:42.676652 27583 tablet_bootstrap.cc:492] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: Bootstrap starting.
I20260925 11:00:42.677290 27584 tablet_bootstrap.cc:492] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: Bootstrap starting.
I20260925 11:00:42.677908 27585 tablet_bootstrap.cc:492] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: Bootstrap starting.
I20260925 11:00:42.678437 27583 tablet_bootstrap.cc:654] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:42.678901 27584 tablet_bootstrap.cc:654] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:42.679602 27585 tablet_bootstrap.cc:654] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:42.681324 27584 tablet_bootstrap.cc:492] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: No bootstrap required, opened a new log
I20260925 11:00:42.681442 27584 ts_tablet_manager.cc:1397] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 11:00:42.681566 27585 tablet_bootstrap.cc:492] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: No bootstrap required, opened a new log
I20260925 11:00:42.681734 27585 ts_tablet_manager.cc:1397] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 11:00:42.682148 27584 raft_consensus.cc:348] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:42.682366 27583 tablet_bootstrap.cc:492] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: No bootstrap required, opened a new log
I20260925 11:00:42.682367 27584 raft_consensus.cc:374] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:42.682454 27584 raft_consensus.cc:729] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd670fbd634028b7f3b5f14401f160, State: Initialized, Role: FOLLOWER
I20260925 11:00:42.682582 27583 ts_tablet_manager.cc:1397] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260925 11:00:42.682492 27585 raft_consensus.cc:348] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:42.682760 27585 raft_consensus.cc:374] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:42.682667 27584 consensus_queue.cc:260] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:42.682857 27585 raft_consensus.cc:729] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869eee58115844f7b37a5c26f25598f2, State: Initialized, Role: FOLLOWER
I20260925 11:00:42.683101 27585 consensus_queue.cc:260] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:42.683239 27584 ts_tablet_manager.cc:1428] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260925 11:00:42.683387 27583 raft_consensus.cc:348] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:42.683653 27583 raft_consensus.cc:374] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:42.683759 27583 raft_consensus.cc:729] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13eb3805718c4b1c81d183e40ae3c8f1, State: Initialized, Role: FOLLOWER
I20260925 11:00:42.684056 27583 consensus_queue.cc:260] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:42.684331 27585 ts_tablet_manager.cc:1428] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260925 11:00:42.684623 27583 ts_tablet_manager.cc:1428] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 11:00:43.017637 27549 raft_consensus.cc:482] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:43.017839 27549 raft_consensus.cc:504] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.018524 27549 leader_election.cc:288] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:43.018985 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54b00ac6eed4b4286db3b7da8747f15" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" is_pre_election: true
I20260925 11:00:43.019061 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54b00ac6eed4b4286db3b7da8747f15" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2" is_pre_election: true
I20260925 11:00:43.019239 27112 raft_consensus.cc:2454] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafd670fbd634028b7f3b5f14401f160 in term 0.
I20260925 11:00:43.019261 27390 raft_consensus.cc:2454] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dafd670fbd634028b7f3b5f14401f160 in term 0.
I20260925 11:00:43.019717 27182 leader_election.cc:302] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1, 869eee58115844f7b37a5c26f25598f2, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:43.019989 27549 raft_consensus.cc:2781] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:43.020098 27549 raft_consensus.cc:482] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:43.020181 27549 raft_consensus.cc:3037] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:43.022280 27549 raft_consensus.cc:504] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.022786 27549 leader_election.cc:288] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 1 election: Requested vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313)
I20260925 11:00:43.023240 27390 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54b00ac6eed4b4286db3b7da8747f15" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "869eee58115844f7b37a5c26f25598f2"
I20260925 11:00:43.023243 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d54b00ac6eed4b4286db3b7da8747f15" candidate_uuid: "dafd670fbd634028b7f3b5f14401f160" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1"
I20260925 11:00:43.023434 27112 raft_consensus.cc:3037] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:43.023433 27390 raft_consensus.cc:3037] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:43.026254 27112 raft_consensus.cc:2454] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dafd670fbd634028b7f3b5f14401f160 in term 1.
I20260925 11:00:43.026254 27390 raft_consensus.cc:2454] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dafd670fbd634028b7f3b5f14401f160 in term 1.
I20260925 11:00:43.026638 27182 leader_election.cc:302] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1, 869eee58115844f7b37a5c26f25598f2, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:43.026917 27549 raft_consensus.cc:2781] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:43.027072 27549 raft_consensus.cc:686] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 1 LEADER]: Becoming Leader. State: Replica: dafd670fbd634028b7f3b5f14401f160, State: Running, Role: LEADER
I20260925 11:00:43.027347 27549 consensus_queue.cc:237] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.029903 26954 catalog_manager.cc:5495] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 reported cstate change: term changed from 0 to 1, leader changed from <none> to dafd670fbd634028b7f3b5f14401f160 (127.24.159.194). New cstate: current_term: 1 leader_uuid: "dafd670fbd634028b7f3b5f14401f160" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } health_report { overall_health: UNKNOWN } } }
I20260925 11:00:43.053030 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:43.047853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44972 (local address 127.24.159.194:39213)
0925 11:00:43.048752 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:43.048757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:43.048773 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:00:43.048818 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:43.048822 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:43.048879 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:43.048959 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:43.049335 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:43.050065 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:43.051423 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:43.051822 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:43.051929 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:43.052348 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:43.052471 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:43.052543 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:43.052682 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 11:00:43.052753 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":789,"thread_start_us":666,"threads_started":1}
I20260925 11:00:43.060485 27390 raft_consensus.cc:1261] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Refusing update from remote peer dafd670fbd634028b7f3b5f14401f160: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:43.060627 27112 raft_consensus.cc:1261] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Refusing update from remote peer dafd670fbd634028b7f3b5f14401f160: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:43.061056 27566 consensus_queue.cc:1035] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [LEADER]: Connected to new peer: Peer: permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:43.061357 27549 consensus_queue.cc:1035] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:43.077098 27554 mvcc.cc:205] Tried to move back new op lower bound from 7333208240371523584 to 7333208240242204672. Current Snapshot: MvccSnapshot[applied={T|T < 7333208240371523584}]
I20260925 11:00:43.106997 26956 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:43.107199 26956 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:43.109850 26956 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043: Sending DeleteTablet for 3 replicas of tablet d54b00ac6eed4b4286db3b7da8747f15
I20260925 11:00:43.110602 27091 tablet_service.cc:1514] Processing DeleteTablet for tablet d54b00ac6eed4b4286db3b7da8747f15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:43 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:44922
I20260925 11:00:43.110641 27230 tablet_service.cc:1514] Processing DeleteTablet for tablet d54b00ac6eed4b4286db3b7da8747f15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:43 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38404
I20260925 11:00:43.110771 27369 tablet_service.cc:1514] Processing DeleteTablet for tablet d54b00ac6eed4b4286db3b7da8747f15 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:43 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:36456
I20260925 11:00:43.110868 27573 tablet_replica.cc:330] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: stopping tablet replica
I20260925 11:00:43.111039 27574 tablet_replica.cc:330] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: stopping tablet replica
I20260925 11:00:43.111060 27573 raft_consensus.cc:2229] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:00:43.111214 27574 raft_consensus.cc:2229] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:43.111388 27573 raft_consensus.cc:2258] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:43.111387 27574 raft_consensus.cc:2258] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:43.111639 27572 tablet_replica.cc:330] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: stopping tablet replica
I20260925 11:00:43.111820 27572 raft_consensus.cc:2229] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:43.111788 26956 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260925 11:00:43.112015 27572 raft_consensus.cc:2258] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260925 11:00:43.112432 26956 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:00:43.112908 27574 ts_tablet_manager.cc:1905] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:43.113281 27572 ts_tablet_manager.cc:1905] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:43.118482 27572 ts_tablet_manager.cc:1918] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:00:43.118773 27574 ts_tablet_manager.cc:1918] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:00:43.118933 27574 log.cc:1196] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/d54b00ac6eed4b4286db3b7da8747f15
I20260925 11:00:43.119228 27091 tablet_service.cc:1467] Processing CreateTablet for tablet 33077528805a41eea166f36b8ed78753 (DEFAULT_TABLE table=test-table [id=df2fec3cc5744f31829f68f2e8a541b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:43.119382 27574 ts_tablet_manager.cc:1939] T d54b00ac6eed4b4286db3b7da8747f15 P 869eee58115844f7b37a5c26f25598f2: Deleting consensus metadata
I20260925 11:00:43.119397 27369 tablet_service.cc:1467] Processing CreateTablet for tablet 33077528805a41eea166f36b8ed78753 (DEFAULT_TABLE table=test-table [id=df2fec3cc5744f31829f68f2e8a541b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:43.119527 27091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33077528805a41eea166f36b8ed78753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:43.119794 27369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33077528805a41eea166f36b8ed78753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:43.120240 27573 ts_tablet_manager.cc:1905] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:43.122336 27572 log.cc:1196] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/d54b00ac6eed4b4286db3b7da8747f15
I20260925 11:00:43.122869 27572 ts_tablet_manager.cc:1939] T d54b00ac6eed4b4286db3b7da8747f15 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting consensus metadata
I20260925 11:00:43.123912 26940 catalog_manager.cc:4841] TS 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939): tablet d54b00ac6eed4b4286db3b7da8747f15 (table test-table [id=88cb5bcf6c3c4a8fa6e974826901b79c]) successfully deleted
I20260925 11:00:43.124223 27583 tablet_bootstrap.cc:492] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: Bootstrap starting.
I20260925 11:00:43.125423 27585 tablet_bootstrap.cc:492] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: Bootstrap starting.
I20260925 11:00:43.125829 27583 tablet_bootstrap.cc:654] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:43.126168 27573 ts_tablet_manager.cc:1918] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:00:43.126322 27573 log.cc:1196] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/d54b00ac6eed4b4286db3b7da8747f15
I20260925 11:00:43.126385 26941 catalog_manager.cc:4841] TS 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313): tablet d54b00ac6eed4b4286db3b7da8747f15 (table test-table [id=88cb5bcf6c3c4a8fa6e974826901b79c]) successfully deleted
I20260925 11:00:43.126691 27573 ts_tablet_manager.cc:1939] T d54b00ac6eed4b4286db3b7da8747f15 P dafd670fbd634028b7f3b5f14401f160: Deleting consensus metadata
I20260925 11:00:43.126925 27585 tablet_bootstrap.cc:654] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:43.127765 26943 catalog_manager.cc:4841] TS dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213): tablet d54b00ac6eed4b4286db3b7da8747f15 (table test-table [id=88cb5bcf6c3c4a8fa6e974826901b79c]) successfully deleted
I20260925 11:00:43.128198 27230 tablet_service.cc:1467] Processing CreateTablet for tablet 33077528805a41eea166f36b8ed78753 (DEFAULT_TABLE table=test-table [id=df2fec3cc5744f31829f68f2e8a541b0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:00:43.128575 27230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33077528805a41eea166f36b8ed78753. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:43.130654 27585 tablet_bootstrap.cc:492] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: No bootstrap required, opened a new log
I20260925 11:00:43.130772 27583 tablet_bootstrap.cc:492] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: No bootstrap required, opened a new log
I20260925 11:00:43.130795 27585 ts_tablet_manager.cc:1397] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260925 11:00:43.131111 27583 ts_tablet_manager.cc:1397] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260925 11:00:43.131479 27585 raft_consensus.cc:348] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.131713 27585 raft_consensus.cc:374] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:43.131788 27585 raft_consensus.cc:729] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 869eee58115844f7b37a5c26f25598f2, State: Initialized, Role: FOLLOWER
I20260925 11:00:43.132011 27585 consensus_queue.cc:260] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.132174 27583 raft_consensus.cc:348] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.132421 27583 raft_consensus.cc:374] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:43.132498 27583 raft_consensus.cc:729] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13eb3805718c4b1c81d183e40ae3c8f1, State: Initialized, Role: FOLLOWER
I20260925 11:00:43.132526 27585 ts_tablet_manager.cc:1428] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 11:00:43.132725 27583 consensus_queue.cc:260] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.133355 27583 ts_tablet_manager.cc:1428] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 11:00:43.135097 27584 tablet_bootstrap.cc:492] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: Bootstrap starting.
I20260925 11:00:43.136423 27584 tablet_bootstrap.cc:654] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:43.138355 27584 tablet_bootstrap.cc:492] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: No bootstrap required, opened a new log
I20260925 11:00:43.138494 27584 ts_tablet_manager.cc:1397] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260925 11:00:43.139042 27584 raft_consensus.cc:348] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.139223 27584 raft_consensus.cc:374] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:43.139298 27584 raft_consensus.cc:729] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dafd670fbd634028b7f3b5f14401f160, State: Initialized, Role: FOLLOWER
I20260925 11:00:43.139505 27584 consensus_queue.cc:260] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.140801 27584 ts_tablet_manager.cc:1428] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 11:00:43.502972 27589 raft_consensus.cc:482] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:00:43.503170 27589 raft_consensus.cc:504] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.503806 27589 leader_election.cc:288] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213)
I20260925 11:00:43.504348 27251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33077528805a41eea166f36b8ed78753" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160" is_pre_election: true
I20260925 11:00:43.504351 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33077528805a41eea166f36b8ed78753" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" is_pre_election: true
I20260925 11:00:43.504573 27251 raft_consensus.cc:2454] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 0.
I20260925 11:00:43.504571 27112 raft_consensus.cc:2454] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 0.
I20260925 11:00:43.504981 27321 leader_election.cc:302] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13eb3805718c4b1c81d183e40ae3c8f1, 869eee58115844f7b37a5c26f25598f2; no voters:
I20260925 11:00:43.505246 27589 raft_consensus.cc:2781] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:00:43.505344 27589 raft_consensus.cc:482] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:00:43.505424 27589 raft_consensus.cc:3037] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:43.507485 27589 raft_consensus.cc:504] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.507995 27589 leader_election.cc:288] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 election: Requested vote from peers 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939), dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213)
I20260925 11:00:43.508425 27112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33077528805a41eea166f36b8ed78753" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13eb3805718c4b1c81d183e40ae3c8f1"
I20260925 11:00:43.508450 27251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33077528805a41eea166f36b8ed78753" candidate_uuid: "869eee58115844f7b37a5c26f25598f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dafd670fbd634028b7f3b5f14401f160"
I20260925 11:00:43.508591 27112 raft_consensus.cc:3037] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:43.508613 27251 raft_consensus.cc:3037] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:43.510596 27112 raft_consensus.cc:2454] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 1.
I20260925 11:00:43.510762 27251 raft_consensus.cc:2454] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 869eee58115844f7b37a5c26f25598f2 in term 1.
I20260925 11:00:43.511157 27324 leader_election.cc:302] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 869eee58115844f7b37a5c26f25598f2, dafd670fbd634028b7f3b5f14401f160; no voters:
I20260925 11:00:43.511404 27589 raft_consensus.cc:2781] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:43.511772 27589 raft_consensus.cc:686] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 1 LEADER]: Becoming Leader. State: Replica: 869eee58115844f7b37a5c26f25598f2, State: Running, Role: LEADER
I20260925 11:00:43.512019 27589 consensus_queue.cc:237] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } }
I20260925 11:00:43.514503 26955 catalog_manager.cc:5495] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 869eee58115844f7b37a5c26f25598f2 (127.24.159.195). New cstate: current_term: 1 leader_uuid: "869eee58115844f7b37a5c26f25598f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "869eee58115844f7b37a5c26f25598f2" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39313 } health_report { overall_health: HEALTHY } } }
I20260925 11:00:43.603407 27604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:43.596835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51328 (local address 127.24.159.195:39313)
0925 11:00:43.598692 (+ 1857us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:43.598696 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:43.598713 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:00:43.598761 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:43.598764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:43.598822 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:43.598967 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:00:43.599543 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:43.600384 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:43.601757 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:43.602154 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:43.602280 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:43.602671 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:00:43.602773 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:00:43.602844 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:43.602977 (+ 133us) server_negotiation.cc:299] Negotiation successful
0925 11:00:43.603049 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1744,"thread_start_us":1620,"threads_started":1}
I20260925 11:00:43.612142 27112 raft_consensus.cc:1261] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Refusing update from remote peer 869eee58115844f7b37a5c26f25598f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:43.612111 27251 raft_consensus.cc:1261] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Refusing update from remote peer 869eee58115844f7b37a5c26f25598f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:00:43.612663 27600 consensus_queue.cc:1035] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dafd670fbd634028b7f3b5f14401f160" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:00:43.612957 27589 consensus_queue.cc:1035] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13eb3805718c4b1c81d183e40ae3c8f1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33939 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 11:00:43.629604 27554 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260925 11:00:45.207767 26955 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:45.208011 26955 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:00:45.211799 26955 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1bc8152db208416aa6e44d9e42fb1043: Sending DeleteTablet for 3 replicas of tablet 33077528805a41eea166f36b8ed78753
I20260925 11:00:45.212607 27091 tablet_service.cc:1514] Processing DeleteTablet for tablet 33077528805a41eea166f36b8ed78753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:45 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:44922
I20260925 11:00:45.212666 27230 tablet_service.cc:1514] Processing DeleteTablet for tablet 33077528805a41eea166f36b8ed78753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:45 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38404
I20260925 11:00:45.212648 27369 tablet_service.cc:1514] Processing DeleteTablet for tablet 33077528805a41eea166f36b8ed78753 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:00:45 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:36456
I20260925 11:00:45.213400 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27027
I20260925 11:00:45.224942 27635 tablet_replica.cc:330] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: stopping tablet replica
I20260925 11:00:45.226918 27633 tablet_replica.cc:330] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: stopping tablet replica
I20260925 11:00:45.227211 27633 raft_consensus.cc:2229] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:45.227521 27633 raft_consensus.cc:2258] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:45.228572 27633 ts_tablet_manager.cc:1905] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:45.236068 27633 ts_tablet_manager.cc:1918] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 11:00:45.236343 27633 log.cc:1196] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/33077528805a41eea166f36b8ed78753
I20260925 11:00:45.236907 27633 ts_tablet_manager.cc:1939] T 33077528805a41eea166f36b8ed78753 P 13eb3805718c4b1c81d183e40ae3c8f1: Deleting consensus metadata
Sep 25 11:00:45 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:00:45.238088 26940 catalog_manager.cc:4841] TS 13eb3805718c4b1c81d183e40ae3c8f1 (127.24.159.193:33939): tablet 33077528805a41eea166f36b8ed78753 (table test-table [id=df2fec3cc5744f31829f68f2e8a541b0]) successfully deleted
I20260925 11:00:45.238391 27635 raft_consensus.cc:2229] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:00:45.238798 27635 raft_consensus.cc:2258] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:45.239336 27634 tablet_replica.cc:330] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: stopping tablet replica
I20260925 11:00:45.239837 27634 raft_consensus.cc:2229] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:00:45.240134 27634 raft_consensus.cc:2258] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:00:45.240630 27635 ts_tablet_manager.cc:1905] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:45.242635 27634 ts_tablet_manager.cc:1905] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:00:45.246057 27637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:45.215228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37670 (local address 127.24.159.193:33939)
0925 11:00:45.226074 (+ 10846us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:45.226080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:45.226106 (+ 26us) server_negotiation.cc:407] Connection header received
0925 11:00:45.226160 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:45.226165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:45.226277 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:45.226378 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:45.228655 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:45.229980 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:45.232003 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:45.232747 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:45.232872 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:45.238612 (+ 5740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:45.238636 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:45.238662 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:45.238708 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:45.241973 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:45.242793 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:45.242802 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:45.242806 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:45.242980 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:45.243532 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:45.243537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:45.243541 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:45.244020 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:45.244287 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:45.245477 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0925 11:00:45.245691 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10714,"thread_start_us":10584,"threads_started":1}
W20260925 11:00:45.246847 27157 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37670
I20260925 11:00:45.247262 27635 ts_tablet_manager.cc:1918] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 11:00:45.247476 27635 log.cc:1196] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/33077528805a41eea166f36b8ed78753
W20260925 11:00:45.247501 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27027: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:45.247737 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 27027
I20260925 11:00:45.248634 27635 ts_tablet_manager.cc:1939] T 33077528805a41eea166f36b8ed78753 P 869eee58115844f7b37a5c26f25598f2: Deleting consensus metadata
I20260925 11:00:45.249722 26941 catalog_manager.cc:4841] TS 869eee58115844f7b37a5c26f25598f2 (127.24.159.195:39313): tablet 33077528805a41eea166f36b8ed78753 (table test-table [id=df2fec3cc5744f31829f68f2e8a541b0]) successfully deleted
I20260925 11:00:45.250468 27634 ts_tablet_manager.cc:1918] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260925 11:00:45.250587 27634 log.cc:1196] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/33077528805a41eea166f36b8ed78753
I20260925 11:00:45.250909 27634 ts_tablet_manager.cc:1939] T 33077528805a41eea166f36b8ed78753 P dafd670fbd634028b7f3b5f14401f160: Deleting consensus metadata
I20260925 11:00:45.251765 26943 catalog_manager.cc:4841] TS dafd670fbd634028b7f3b5f14401f160 (127.24.159.194:39213): tablet 33077528805a41eea166f36b8ed78753 (table test-table [id=df2fec3cc5744f31829f68f2e8a541b0]) successfully deleted
I20260925 11:00:45.267162 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27166
Sep 25 11:00:45 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:00:45.282060 27638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:45.267852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44984 (local address 127.24.159.194:39213)
0925 11:00:45.268358 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:45.268363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:45.268379 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:00:45.268433 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:45.268437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:45.268502 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:45.268594 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:45.269826 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:45.270629 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:45.271812 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:45.272270 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:45.272379 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:45.276397 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:45.276413 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:45.276430 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:45.276468 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:45.279288 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:45.280060 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:45.280068 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:45.280070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:45.280230 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:45.280648 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:45.280651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:45.280654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:45.281047 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:45.281288 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:45.281556 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 11:00:45.281728 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":242,"threads_started":1}
W20260925 11:00:45.282757 27296 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44984
W20260925 11:00:45.283438 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27166: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:45.283577 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 27166
I20260925 11:00:45.301898 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27305
Sep 25 11:00:45 dist-test-slave-trms krb5kdc[26911](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334042, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:00:45.315793 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:45.302606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51330 (local address 127.24.159.195:39313)
0925 11:00:45.303059 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:45.303063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:45.303083 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:00:45.303138 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:45.303141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:45.303204 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:45.303289 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:45.304444 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:45.305130 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:45.306345 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:45.306737 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:45.306829 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:45.310866 (+ 4037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:45.310880 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:45.310896 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:45.310928 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:45.313512 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:45.314062 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:45.314068 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:45.314070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:45.314221 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:45.314521 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:45.314524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:45.314526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:45.314866 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:45.315074 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:45.315280 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 11:00:45.315445 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":229,"threads_started":1}
W20260925 11:00:45.316452 27435 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51330
W20260925 11:00:45.317052 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27305: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:45.317214 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 27305
I20260925 11:00:45.335160 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 26924
I20260925 11:00:45.336282 26996 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:45.450738 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 26924
2026-09-25T11:00:45Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (10483 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:45 dist-test-slave-trms krb5kdc[27643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:45 dist-test-slave-trms krb5kdc[27643](info): set up 2 sockets
Sep 25 11:00:45 dist-test-slave-trms krb5kdc[27643](info): commencing operation
krb5kdc: starting...
W20260925 11:00:46.442608 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.952s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:00:46 dist-test-slave-trms krb5kdc[27643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:00:46.617204 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:38137
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:38137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:46.924230 27656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:46.924517 27656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:46.924644 27656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:46.931915 27656 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:46.932024 27656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:46.932098 27656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:46.932178 27656 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:46.932233 27656 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:46.942081 27656 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:38137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:38137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:46.943446 27656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:46.944782 27656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:46.956281 27661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:46.956311 27662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:46.956274 27664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:46.957041 27656 server_base.cc:1034] running on GCE node
I20260925 11:00:46.957814 27656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:46.959276 27656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:46.960481 27656 hybrid_clock.cc:648] HybridClock initialized: now 1790334046960439 us; error 32 us; skew 500 ppm
I20260925 11:00:46.960813 27656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:46 dist-test-slave-trms krb5kdc[27643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334046, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:46.963730 27656 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:00:46.965137 27656 webserver.cc:460] Webserver started at http://127.24.159.254:37153/ using document root <none> and password file <none>
I20260925 11:00:46.965793 27656 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:46.965895 27656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:46.966150 27656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:46.969084 27656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4"
format_stamp: "Formatted at 2026-09-25 11:00:46 on dist-test-slave-trms"
server_key: "986950de6e207b5a9a131745dbe518f3"
server_key_iv: "57dd6de14bc8185f2f612719017bdd99"
server_key_version: "encryptionkey@0"
I20260925 11:00:46.969789 27656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4"
format_stamp: "Formatted at 2026-09-25 11:00:46 on dist-test-slave-trms"
server_key: "986950de6e207b5a9a131745dbe518f3"
server_key_iv: "57dd6de14bc8185f2f612719017bdd99"
server_key_version: "encryptionkey@0"
I20260925 11:00:46.975101 27656 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260925 11:00:46.978614 27671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:46.979985 27656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:00:46.980135 27656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4"
format_stamp: "Formatted at 2026-09-25 11:00:46 on dist-test-slave-trms"
server_key: "986950de6e207b5a9a131745dbe518f3"
server_key_iv: "57dd6de14bc8185f2f612719017bdd99"
server_key_version: "encryptionkey@0"
I20260925 11:00:46.980307 27656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:46.999294 27656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:47.003574 27656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:47.003826 27656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:47.019831 27656 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:38137
I20260925 11:00:47.019833 27722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:38137 every 8 connection(s)
I20260925 11:00:47.021199 27656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:47.022485 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 27656
I20260925 11:00:47.022703 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:00:47.023464 25215 external_mini_cluster.cc:1428] Setting key b2437af4440a5170b0393d6ff1cf32d9
I20260925 11:00:47.025843 27723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 11:00:47 dist-test-slave-trms krb5kdc[27643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:47.038297 27723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4: Bootstrap starting.
I20260925 11:00:47.042320 27723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:47.043469 27723 log.cc:824] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:47.046439 27726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:47.025438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39726 (local address 127.24.159.254:38137)
0925 11:00:47.026179 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:47.026221 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:47.026272 (+ 51us) server_negotiation.cc:407] Connection header received
0925 11:00:47.027881 (+ 1609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:47.027901 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:47.028300 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:47.028621 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:47.030334 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:47.031641 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:47.033307 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:47.033956 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:47.034077 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:47.038442 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:47.038453 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:47.038472 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:47.038521 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:47.042112 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:47.042831 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:47.042838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:47.042846 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:47.043053 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:47.043354 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:47.043358 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:47.043361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:47.044059 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:47.044308 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:47.044709 (+ 401us) server_negotiation.cc:299] Negotiation successful
0925 11:00:47.045142 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":335,"threads_started":1}
I20260925 11:00:47.046747 27723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4: No bootstrap required, opened a new log
I20260925 11:00:47.054399 27723 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38137 } }
I20260925 11:00:47.054715 27723 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:47.054787 27723 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb253b93fc8d4fe0b4884acc2d1dbfb4, State: Initialized, Role: FOLLOWER
I20260925 11:00:47.055279 27723 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38137 } }
I20260925 11:00:47.055480 27723 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:47.055573 27723 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:47.055696 27723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:47.057677 27723 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38137 } }
I20260925 11:00:47.058097 27723 leader_election.cc:302] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb253b93fc8d4fe0b4884acc2d1dbfb4; no voters:
I20260925 11:00:47.058652 27723 leader_election.cc:288] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:47.058777 27728 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:47.059345 27728 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [term 1 LEADER]: Becoming Leader. State: Replica: bb253b93fc8d4fe0b4884acc2d1dbfb4, State: Running, Role: LEADER
I20260925 11:00:47.059852 27728 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38137 } }
I20260925 11:00:47.060563 27723 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:47.062753 27729 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38137 } } }
I20260925 11:00:47.062736 27730 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb253b93fc8d4fe0b4884acc2d1dbfb4. Latest consensus state: current_term: 1 leader_uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb253b93fc8d4fe0b4884acc2d1dbfb4" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38137 } } }
I20260925 11:00:47.063656 27730 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:47.064205 27729 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:47.064855 27736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:47.069833 27736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:47.078341 27736 catalog_manager.cc:1347] Generated new cluster ID: 4b64b4043a4c469aa35aef9ce1d19d40
I20260925 11:00:47.078435 27736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:47.100859 27736 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:47.102732 27736 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:47.113807 27736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb253b93fc8d4fe0b4884acc2d1dbfb4: Generated new TSK 0
I20260925 11:00:47.114889 27736 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:00:47.157606 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:38137
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:47.488657 27749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:47.488936 27749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:47.489035 27749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:47.495924 27749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:47.496024 27749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:47.496136 27749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:00:47.505496 27749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:38137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:47.506774 27749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:47.508109 27749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:47.520479 27754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:47.520537 27755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:47.520519 27757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:47.521292 27749 server_base.cc:1034] running on GCE node
I20260925 11:00:47.521855 27749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:47.522799 27749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:47.524021 27749 hybrid_clock.cc:648] HybridClock initialized: now 1790334047523968 us; error 40 us; skew 500 ppm
I20260925 11:00:47.524333 27749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:47 dist-test-slave-trms krb5kdc[27643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334047, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:47.527290 27749 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:00:47.528662 27749 webserver.cc:460] Webserver started at http://127.24.159.193:37599/ using document root <none> and password file <none>
I20260925 11:00:47.529317 27749 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:47.529420 27749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:47.529664 27749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:47.532501 27749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "a76ce742bf4245249d616b86ce5aa9f6"
format_stamp: "Formatted at 2026-09-25 11:00:47 on dist-test-slave-trms"
server_key: "d98fb4f02a0de0c1e904540f0441cb68"
server_key_iv: "ca5d4dedf3f95c2c4e129c9e2e76e010"
server_key_version: "encryptionkey@0"
I20260925 11:00:47.533211 27749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "a76ce742bf4245249d616b86ce5aa9f6"
format_stamp: "Formatted at 2026-09-25 11:00:47 on dist-test-slave-trms"
server_key: "d98fb4f02a0de0c1e904540f0441cb68"
server_key_iv: "ca5d4dedf3f95c2c4e129c9e2e76e010"
server_key_version: "encryptionkey@0"
I20260925 11:00:47.538115 27749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 11:00:47.541404 27764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:47.542729 27749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 11:00:47.542869 27749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "a76ce742bf4245249d616b86ce5aa9f6"
format_stamp: "Formatted at 2026-09-25 11:00:47 on dist-test-slave-trms"
server_key: "d98fb4f02a0de0c1e904540f0441cb68"
server_key_iv: "ca5d4dedf3f95c2c4e129c9e2e76e010"
server_key_version: "encryptionkey@0"
I20260925 11:00:47.543037 27749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:47.563618 27749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:47.567709 27749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:47.567953 27749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:47.568852 27749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:47.570118 27749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:47.570225 27749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:47.570343 27749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:47.570399 27749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:47.598397 27749 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:39991
I20260925 11:00:47.598429 27876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:39991 every 8 connection(s)
I20260925 11:00:47.599614 27749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:00:47.606261 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 27749
I20260925 11:00:47.606482 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:00:47.607208 25215 external_mini_cluster.cc:1428] Setting key f3a59eda0027caebc32e7e252e6be142
Sep 25 11:00:47 dist-test-slave-trms krb5kdc[27643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334047, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:47.620175 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:47.601975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:53199 (local address 127.24.159.254:38137)
0925 11:00:47.602384 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:47.602388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:47.603318 (+ 930us) server_negotiation.cc:407] Connection header received
0925 11:00:47.605109 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:47.605115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:47.605204 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:47.605298 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:47.607303 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:47.608062 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:47.610258 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:47.610665 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:47.610775 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:47.614635 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:47.614643 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:47.614647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:47.614685 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:47.617280 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:47.617945 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:47.617948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:47.617951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:47.618061 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:47.618508 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:47.618531 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:47.618533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:47.618802 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:47.618927 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:47.619616 (+ 689us) server_negotiation.cc:299] Negotiation successful
0925 11:00:47.619845 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":221,"threads_started":1}
I20260925 11:00:47.621147 27880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:47.602486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:38137 (local address 127.24.159.193:53199)
0925 11:00:47.603158 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:47.603198 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:47.604850 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:47.605546 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:47.605555 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:47.606304 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:47.607110 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:47.607134 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:47.608238 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:47.608243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:47.610074 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:47.610088 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:47.610894 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:47.610901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:47.611172 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:47.611833 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:47.611860 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:47.614403 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:47.617406 (+ 3003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:47.617415 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:47.617431 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:47.617828 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:47.618155 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:47.618159 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:47.618163 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:47.618391 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:47.618916 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:47.618923 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:47.619250 (+ 327us) client_negotiation.cc:769] Sending connection context
0925 11:00:47.619483 (+ 233us) client_negotiation.cc:240] Negotiation successful
0925 11:00:47.619983 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":259,"threads_started":1}
I20260925 11:00:47.622601 27877 heartbeater.cc:347] Connected to a master server at 127.24.159.254:38137
I20260925 11:00:47.622882 27877 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:47.623548 27877 heartbeater.cc:511] Master 127.24.159.254:38137 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
I20260925 11:00:47.625495 27688 ts_manager.cc:194] Registered new tserver with Master: a76ce742bf4245249d616b86ce5aa9f6 (127.24.159.193:39991)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
I20260925 11:00:47.626938 27688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:53199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:00:47.645498 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:38137
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:47.947640 27883 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:47.947959 27883 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:47.948064 27883 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:47.955056 27883 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:47.955155 27883 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:47.955273 27883 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:00:47.964673 27883 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:38137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:47.965903 27883 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:47.967175 27883 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:47.979421 27889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:47.979442 27891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:47.979398 27888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:47.980104 27883 server_base.cc:1034] running on GCE node
I20260925 11:00:47.980693 27883 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:47.981971 27883 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:47.983197 27883 hybrid_clock.cc:648] HybridClock initialized: now 1790334047983117 us; error 70 us; skew 500 ppm
I20260925 11:00:47.983598 27883 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:47 dist-test-slave-trms krb5kdc[27643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334047, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:47.986622 27883 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:00:47.987932 27883 webserver.cc:460] Webserver started at http://127.24.159.194:45133/ using document root <none> and password file <none>
I20260925 11:00:47.988624 27883 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:47.988737 27883 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:47.989012 27883 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:47.991891 27883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "469e22a207134fa9bae2e2c93fb3c595"
format_stamp: "Formatted at 2026-09-25 11:00:47 on dist-test-slave-trms"
server_key: "09fda926a508505bc25789a53234cd90"
server_key_iv: "aad172bfdf86acb53a3e5f69d51fafb9"
server_key_version: "encryptionkey@0"
I20260925 11:00:47.992624 27883 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "469e22a207134fa9bae2e2c93fb3c595"
format_stamp: "Formatted at 2026-09-25 11:00:47 on dist-test-slave-trms"
server_key: "09fda926a508505bc25789a53234cd90"
server_key_iv: "aad172bfdf86acb53a3e5f69d51fafb9"
server_key_version: "encryptionkey@0"
I20260925 11:00:47.997563 27883 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260925 11:00:48.000942 27898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:48.002261 27883 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:00:48.002408 27883 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "469e22a207134fa9bae2e2c93fb3c595"
format_stamp: "Formatted at 2026-09-25 11:00:47 on dist-test-slave-trms"
server_key: "09fda926a508505bc25789a53234cd90"
server_key_iv: "aad172bfdf86acb53a3e5f69d51fafb9"
server_key_version: "encryptionkey@0"
I20260925 11:00:48.002599 27883 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:48.016762 27883 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:48.020679 27883 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:48.020934 27883 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:48.021859 27883 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:48.023285 27883 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:48.023356 27883 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:48.023463 27883 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:48.023516 27883 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:48.058713 27883 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:39975
I20260925 11:00:48.058748 28010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:39975 every 8 connection(s)
I20260925 11:00:48.059965 27883 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:00:48.069778 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 27883
I20260925 11:00:48.069948 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:00:48.070501 25215 external_mini_cluster.cc:1428] Setting key 23d7830c8f227a71e87da38f181ee7ba
Sep 25 11:00:48 dist-test-slave-trms krb5kdc[27643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334047, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:48.079671 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:48.062398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:35087 (local address 127.24.159.254:38137)
0925 11:00:48.062565 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:48.062570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:48.063742 (+ 1172us) server_negotiation.cc:407] Connection header received
0925 11:00:48.065605 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:48.065610 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:48.065694 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:48.065780 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:48.067346 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:48.068063 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:48.069576 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:48.070028 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:48.070126 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:48.074227 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:48.074234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:48.074238 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:48.074268 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:48.076653 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:48.077469 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:48.077473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:48.077475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:48.077598 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:48.078002 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:48.078006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:48.078008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:48.078304 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:48.078442 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:48.079111 (+ 669us) server_negotiation.cc:299] Negotiation successful
0925 11:00:48.079339 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 11:00:48.080611 28013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:48.062900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:38137 (local address 127.24.159.194:35087)
0925 11:00:48.063587 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:48.063625 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:48.065323 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:48.066031 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:48.066043 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:48.066626 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:48.067185 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:48.067202 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:48.068210 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:48.068214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:48.069435 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:48.069450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:48.070284 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:48.070292 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:48.070605 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:48.071261 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:48.071291 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:48.073994 (+ 2703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:48.076788 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:48.076796 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:48.076813 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:48.077342 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:48.077701 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:48.077706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:48.077709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:48.077892 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:48.078417 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:48.078425 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:48.078757 (+ 332us) client_negotiation.cc:769] Sending connection context
0925 11:00:48.078990 (+ 233us) client_negotiation.cc:240] Negotiation successful
0925 11:00:48.079498 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":254,"threads_started":1}
I20260925 11:00:48.082011 28011 heartbeater.cc:347] Connected to a master server at 127.24.159.254:38137
I20260925 11:00:48.082329 28011 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:48.083027 28011 heartbeater.cc:511] Master 127.24.159.254:38137 requested a full tablet report, sending...
I20260925 11:00:48.084575 27688 ts_manager.cc:194] Registered new tserver with Master: 469e22a207134fa9bae2e2c93fb3c595 (127.24.159.194:39975)
I20260925 11:00:48.085691 27688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:35087
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:00:48.108378 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:38137
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:48.416575 28016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:48.416852 28016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:48.416970 28016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:48.424024 28016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:48.424134 28016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:48.424252 28016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:00:48.433578 28016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:38137
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:48.434813 28016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:48.436072 28016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:48.448345 28022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:48.448446 28021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:48.448432 28024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:48.449003 28016 server_base.cc:1034] running on GCE node
I20260925 11:00:48.449649 28016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:48.450587 28016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:48.451783 28016 hybrid_clock.cc:648] HybridClock initialized: now 1790334048451734 us; error 44 us; skew 500 ppm
I20260925 11:00:48.452077 28016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:48 dist-test-slave-trms krb5kdc[27643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334048, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:48.454969 28016 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:00:48.456290 28016 webserver.cc:460] Webserver started at http://127.24.159.195:43383/ using document root <none> and password file <none>
I20260925 11:00:48.456915 28016 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:48.457010 28016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:48.457247 28016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:48.460096 28016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "a63f073aa63340c3918df8dfa356cbad"
format_stamp: "Formatted at 2026-09-25 11:00:48 on dist-test-slave-trms"
server_key: "b558dc1c8e5b09ae8c398f4fde4a23db"
server_key_iv: "3c38713c8609e1d84257b997b6b85d43"
server_key_version: "encryptionkey@0"
I20260925 11:00:48.460825 28016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "a63f073aa63340c3918df8dfa356cbad"
format_stamp: "Formatted at 2026-09-25 11:00:48 on dist-test-slave-trms"
server_key: "b558dc1c8e5b09ae8c398f4fde4a23db"
server_key_iv: "3c38713c8609e1d84257b997b6b85d43"
server_key_version: "encryptionkey@0"
I20260925 11:00:48.465844 28016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 11:00:48.469513 28031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:48.471287 28016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 11:00:48.471437 28016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "a63f073aa63340c3918df8dfa356cbad"
format_stamp: "Formatted at 2026-09-25 11:00:48 on dist-test-slave-trms"
server_key: "b558dc1c8e5b09ae8c398f4fde4a23db"
server_key_iv: "3c38713c8609e1d84257b997b6b85d43"
server_key_version: "encryptionkey@0"
I20260925 11:00:48.471617 28016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:48.493057 28016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:48.497120 28016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:48.497368 28016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:48.498266 28016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:48.499505 28016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:48.499580 28016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:48.499689 28016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:48.499735 28016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:48.527660 28016 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:46813
I20260925 11:00:48.527707 28143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:46813 every 8 connection(s)
I20260925 11:00:48.528931 28016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:00:48.532840 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 28016
I20260925 11:00:48.533022 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:00:48.533565 25215 external_mini_cluster.cc:1428] Setting key 9f72f636a4712384a613a565f46009f1
Sep 25 11:00:48 dist-test-slave-trms krb5kdc[27643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334048, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:48.549290 27879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:48.531533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:36305 (local address 127.24.159.254:38137)
0925 11:00:48.531704 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:48.531710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:48.532938 (+ 1228us) server_negotiation.cc:407] Connection header received
0925 11:00:48.535271 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:48.535276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:48.535361 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:48.535446 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:48.536970 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:48.537691 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:48.539222 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:48.539598 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:48.539704 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:48.542991 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:48.542996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:48.542999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:48.543030 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:48.545738 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:48.546551 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:48.546555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:48.546557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:48.546680 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:48.547088 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:48.547092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:48.547094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:48.547452 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:48.547580 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:48.548758 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0925 11:00:48.548960 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:00:48.549940 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:48.531988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:38137 (local address 127.24.159.195:36305)
0925 11:00:48.532739 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:48.532795 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:48.535000 (+ 2205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:48.535687 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:48.535704 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:48.536187 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:48.536820 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:48.536837 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:48.537823 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:48.537828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:48.539086 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:48.539098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:48.539819 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:48.539826 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:48.540044 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:48.540701 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:48.540729 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:48.542800 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:48.545880 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:48.545890 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:48.545906 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:48.546408 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:48.546786 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:48.546791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:48.546795 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:48.546979 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:48.547568 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:48.547576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:48.547906 (+ 330us) client_negotiation.cc:769] Sending connection context
0925 11:00:48.548165 (+ 259us) client_negotiation.cc:240] Negotiation successful
0925 11:00:48.548641 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":245,"threads_started":1}
I20260925 11:00:48.551199 28144 heartbeater.cc:347] Connected to a master server at 127.24.159.254:38137
I20260925 11:00:48.551446 28144 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:48.552079 28144 heartbeater.cc:511] Master 127.24.159.254:38137 requested a full tablet report, sending...
I20260925 11:00:48.553371 27688 ts_manager.cc:194] Registered new tserver with Master: a63f073aa63340c3918df8dfa356cbad (127.24.159.195:46813)
I20260925 11:00:48.554150 27688 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:36305
I20260925 11:00:48.560371 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 11:00:48.591799 27879 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:48.589435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39730 (local address 127.24.159.254:38137)
0925 11:00:48.589642 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:48.589649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:48.589983 (+ 334us) server_negotiation.cc:407] Connection header received
0925 11:00:48.590142 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:48.590147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:48.590255 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:48.590330 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:48.591540 (+ 1210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39730: BlockingRecv error: recv got EOF from 127.0.0.1:39730 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:00:48.595162 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27749
W20260925 11:00:48.597638 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27749: Not authorized: Client connection negotiation failed: client connection to 127.24.159.193:39991: server requires authentication, but client does not have Kerberos credentials available
I20260925 11:00:48.597805 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 27749
W20260925 11:00:48.597909 28155 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:00:48.596021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45858 (local address 127.24.159.193:39991)
0925 11:00:48.596567 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:48.596574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:48.596600 (+ 26us) server_negotiation.cc:407] Connection header received
0925 11:00:48.596658 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:48.596664 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:48.596880 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:48.597028 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:48.597575 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45858: BlockingRecv error: recv got EOF from 127.0.0.1:45858 (error 108)
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":252,"threads_started":1}
I20260925 11:00:48.610615 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27883
W20260925 11:00:48.613055 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27883: Not authorized: Client connection negotiation failed: client connection to 127.24.159.194:39975: server requires authentication, but client does not have Kerberos credentials available
I20260925 11:00:48.613219 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 27883
I20260925 11:00:48.625706 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28016
W20260925 11:00:48.628473 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28016: Not authorized: Client connection negotiation failed: client connection to 127.24.159.195:46813: server requires authentication, but client does not have Kerberos credentials available
I20260925 11:00:48.628595 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 28016
I20260925 11:00:48.640762 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 27656
I20260925 11:00:48.641954 27718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:48.744621 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 27656
2026-09-25T11:00:48Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3291 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:49 dist-test-slave-trms krb5kdc[28161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:49 dist-test-slave-trms krb5kdc[28161](info): set up 2 sockets
Sep 25 11:00:49 dist-test-slave-trms krb5kdc[28161](info): commencing operation
krb5kdc: starting...
W20260925 11:00:50.530573 25215 mini_kdc.cc:121] Time spent starting KDC: real 1.749s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:00:50 dist-test-slave-trms krb5kdc[28161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:00:50.701793 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:46861
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:46861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:51.012553 28174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:51.012840 28174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:51.012966 28174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:51.020107 28174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:51.020220 28174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:51.020292 28174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:51.020373 28174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:51.020437 28174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:51.030375 28174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:46861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:46861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:51.031672 28174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:51.032980 28174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:51.043977 28182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:51.044039 28179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:51.044039 28180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:51.044776 28174 server_base.cc:1034] running on GCE node
I20260925 11:00:51.045521 28174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:51.046932 28174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:51.048130 28174 hybrid_clock.cc:648] HybridClock initialized: now 1790334051048080 us; error 39 us; skew 500 ppm
I20260925 11:00:51.048465 28174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:51 dist-test-slave-trms krb5kdc[28161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334051, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:51.054658 28174 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:00:51.056173 28174 webserver.cc:460] Webserver started at http://127.24.159.254:33497/ using document root <none> and password file <none>
I20260925 11:00:51.056875 28174 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:51.056995 28174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:51.057289 28174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:51.060353 28174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45"
format_stamp: "Formatted at 2026-09-25 11:00:51 on dist-test-slave-trms"
server_key: "f85c880b29bb262bda10b699ed7c4255"
server_key_iv: "f08c166dcc9aa37bb883b6348c448883"
server_key_version: "encryptionkey@0"
I20260925 11:00:51.061152 28174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45"
format_stamp: "Formatted at 2026-09-25 11:00:51 on dist-test-slave-trms"
server_key: "f85c880b29bb262bda10b699ed7c4255"
server_key_iv: "f08c166dcc9aa37bb883b6348c448883"
server_key_version: "encryptionkey@0"
I20260925 11:00:51.066421 28174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260925 11:00:51.069824 28189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:51.071199 28174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:51.071352 28174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45"
format_stamp: "Formatted at 2026-09-25 11:00:51 on dist-test-slave-trms"
server_key: "f85c880b29bb262bda10b699ed7c4255"
server_key_iv: "f08c166dcc9aa37bb883b6348c448883"
server_key_version: "encryptionkey@0"
I20260925 11:00:51.071524 28174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:51.087420 28174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:51.091228 28174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:51.091477 28174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:51.107954 28174 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:46861
I20260925 11:00:51.107986 28240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:46861 every 8 connection(s)
I20260925 11:00:51.109273 28174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:51.112882 28241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:51.116904 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 28174
I20260925 11:00:51.117136 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:00:51.117913 25215 external_mini_cluster.cc:1428] Setting key d276a22103910c01f03a9cb3c756687f
I20260925 11:00:51.124284 28241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45: Bootstrap starting.
I20260925 11:00:51.127878 28241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:51.129060 28241 log.cc:824] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:51.132324 28241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45: No bootstrap required, opened a new log
Sep 25 11:00:51 dist-test-slave-trms krb5kdc[28161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334050, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:51.141836 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:51.119783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.24.159.254:46861)
0925 11:00:51.120498 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:51.120510 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:51.120551 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:00:51.122766 (+ 2215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:51.122789 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:51.123200 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:51.123515 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:51.125243 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:51.126619 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:51.128297 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:51.129006 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:51.129140 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:51.133492 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:51.133503 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:51.133524 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:51.133569 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:51.137312 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:51.137971 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:51.137979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:51.137989 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:51.138225 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:51.138559 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:51.138564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:51.138568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:51.139327 (+ 759us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:51.139638 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:51.139937 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 11:00:51.140413 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":312,"threads_started":1}
I20260925 11:00:51.142575 28241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46861 } }
I20260925 11:00:51.142943 28241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:51.143036 28241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aa1b5ccdfc4fce9ce76ed899c3bc45, State: Initialized, Role: FOLLOWER
I20260925 11:00:51.143685 28241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46861 } }
I20260925 11:00:51.143963 28241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:51.144104 28241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:51.144270 28241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:51.145768 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:00:51.146730 28241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46861 } }
I20260925 11:00:51.147195 28241 leader_election.cc:302] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15aa1b5ccdfc4fce9ce76ed899c3bc45; no voters:
I20260925 11:00:51.147792 28241 leader_election.cc:288] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:51.147898 28246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:51.150144 28241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:51.154778 28246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [term 1 LEADER]: Becoming Leader. State: Replica: 15aa1b5ccdfc4fce9ce76ed899c3bc45, State: Running, Role: LEADER
I20260925 11:00:51.155229 28246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46861 } }
I20260925 11:00:51.157770 28248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15aa1b5ccdfc4fce9ce76ed899c3bc45. Latest consensus state: current_term: 1 leader_uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46861 } } }
I20260925 11:00:51.158218 28248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:51.157784 28247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa1b5ccdfc4fce9ce76ed899c3bc45" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46861 } } }
I20260925 11:00:51.158713 28247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45 [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:51.165066 28262 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 11:00:51.167017 28261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:51.173488 28261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:51.183390 28261 catalog_manager.cc:1347] Generated new cluster ID: 4ec64d49922e4cd9bb83089497471202
I20260925 11:00:51.183574 28261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:51.187819 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:51.180222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32978 (local address 127.24.159.254:46861)
0925 11:00:51.180382 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:51.180387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:51.180531 (+ 144us) server_negotiation.cc:407] Connection header received
0925 11:00:51.180746 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:51.180752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:51.180836 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:51.180945 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:51.182347 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:51.183450 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:51.184775 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:51.185241 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:51.185355 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:51.186409 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:51.186414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:51.186417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:00:51.187018 (+ 601us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:51.187080 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:00:51.187107 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:51.187200 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:51.187346 (+ 146us) server_negotiation.cc:299] Negotiation successful
0925 11:00:51.187440 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":40}
I20260925 11:00:51.217888 28261 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:51.219825 28261 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:51.232255 28261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15aa1b5ccdfc4fce9ce76ed899c3bc45: Generated new TSK 0
I20260925 11:00:51.233369 28261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:00:51.250344 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28174
I20260925 11:00:51.251657 28236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:51.341514 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 28174
2026-09-25T11:00:51Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2591 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:51 dist-test-slave-trms krb5kdc[28275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:51 dist-test-slave-trms krb5kdc[28275](info): set up 2 sockets
Sep 25 11:00:51 dist-test-slave-trms krb5kdc[28275](info): commencing operation
krb5kdc: starting...
W20260925 11:00:52.334628 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.961s user 0.004s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:00:52 dist-test-slave-trms krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:00:52.511950 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44535
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:46189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:44535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:52.826867 28288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:52.827169 28288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:52.827306 28288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:52.834609 28288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:52.834736 28288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:52.834813 28288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:52.834889 28288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:52.834949 28288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:52.844748 28288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:46189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:44535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:52.846065 28288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:52.847393 28288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:52.858397 28293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:52.858465 28296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:52.858850 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:52.859643 28288 server_base.cc:1034] running on GCE node
I20260925 11:00:52.860351 28288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:52.861728 28288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:52.862931 28288 hybrid_clock.cc:648] HybridClock initialized: now 1790334052862883 us; error 33 us; skew 500 ppm
I20260925 11:00:52.863245 28288 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:52 dist-test-slave-trms krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334052, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:52.866096 28288 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:00:52.867512 28288 webserver.cc:460] Webserver started at http://127.24.159.254:37169/ using document root <none> and password file <none>
I20260925 11:00:52.868146 28288 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:52.868243 28288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:52.868482 28288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:52.871402 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "2d389f32e67648d3bfc3e0581b11da9b"
format_stamp: "Formatted at 2026-09-25 11:00:52 on dist-test-slave-trms"
server_key: "a770056c759b2a0385189cacbaa8840c"
server_key_iv: "9e938eea87c06f24ba67af3b4b89d1bd"
server_key_version: "encryptionkey@0"
I20260925 11:00:52.872222 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "2d389f32e67648d3bfc3e0581b11da9b"
format_stamp: "Formatted at 2026-09-25 11:00:52 on dist-test-slave-trms"
server_key: "a770056c759b2a0385189cacbaa8840c"
server_key_iv: "9e938eea87c06f24ba67af3b4b89d1bd"
server_key_version: "encryptionkey@0"
I20260925 11:00:52.877370 28288 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 11:00:52.880671 28303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:52.881991 28288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:00:52.882150 28288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "2d389f32e67648d3bfc3e0581b11da9b"
format_stamp: "Formatted at 2026-09-25 11:00:52 on dist-test-slave-trms"
server_key: "a770056c759b2a0385189cacbaa8840c"
server_key_iv: "9e938eea87c06f24ba67af3b4b89d1bd"
server_key_version: "encryptionkey@0"
I20260925 11:00:52.882390 28288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:52.895687 28288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:52.899356 28288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:52.899600 28288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:52.916927 28288 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:44535
I20260925 11:00:52.916942 28354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:44535 every 8 connection(s)
I20260925 11:00:52.918560 28288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:52.923290 28355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:52.928154 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 28288
I20260925 11:00:52.928335 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:00:52.928941 25215 external_mini_cluster.cc:1428] Setting key 8d5a2f465fb10029af32b6869082ae26
I20260925 11:00:52.934163 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b: Bootstrap starting.
I20260925 11:00:52.937318 28355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:52.938338 28355 log.cc:824] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:52.941102 28355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b: No bootstrap required, opened a new log
Sep 25 11:00:52 dist-test-slave-trms krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334052, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:52.948812 28355 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d389f32e67648d3bfc3e0581b11da9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44535 } }
I20260925 11:00:52.949215 28355 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:52.949318 28355 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d389f32e67648d3bfc3e0581b11da9b, State: Initialized, Role: FOLLOWER
I20260925 11:00:52.949921 28355 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d389f32e67648d3bfc3e0581b11da9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44535 } }
I20260925 11:00:52.950095 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:52.930941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.24.159.254:44535)
0925 11:00:52.931761 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:52.931774 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:52.931818 (+ 44us) server_negotiation.cc:407] Connection header received
0925 11:00:52.933467 (+ 1649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:52.933489 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:52.933886 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:52.934276 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:52.935579 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:52.936865 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:52.938126 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:52.938955 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:52.939135 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:52.943015 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:52.943023 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:52.943040 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:52.943076 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:52.946003 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:52.946702 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:52.946709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:52.946716 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:52.946886 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:52.947267 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:52.947271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:52.947273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:52.947854 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:52.948144 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:52.948566 (+ 422us) server_negotiation.cc:299] Negotiation successful
0925 11:00:52.949007 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":353,"threads_started":1}
I20260925 11:00:52.950145 28355 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:52.950284 28355 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:52.950495 28355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:52.952733 28355 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d389f32e67648d3bfc3e0581b11da9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44535 } }
I20260925 11:00:52.953234 28355 leader_election.cc:302] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d389f32e67648d3bfc3e0581b11da9b; no voters:
I20260925 11:00:52.953754 28355 leader_election.cc:288] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:52.953873 28360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:52.954480 28360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [term 1 LEADER]: Becoming Leader. State: Replica: 2d389f32e67648d3bfc3e0581b11da9b, State: Running, Role: LEADER
I20260925 11:00:52.954938 28360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d389f32e67648d3bfc3e0581b11da9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44535 } }
I20260925 11:00:52.955636 28355 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:52.957415 28362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d389f32e67648d3bfc3e0581b11da9b. Latest consensus state: current_term: 1 leader_uuid: "2d389f32e67648d3bfc3e0581b11da9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d389f32e67648d3bfc3e0581b11da9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44535 } } }
I20260925 11:00:52.957770 28362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:52.959043 28361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d389f32e67648d3bfc3e0581b11da9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d389f32e67648d3bfc3e0581b11da9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44535 } } }
I20260925 11:00:52.959416 28361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:52.963294 28367 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:52.968585 28367 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:52.977110 28367 catalog_manager.cc:1347] Generated new cluster ID: 3c0b7816aaa64d92b67e1a9cda16ddd1
I20260925 11:00:52.977218 28367 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:52.990955 28367 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:52.992249 28367 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:53.007750 28367 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d389f32e67648d3bfc3e0581b11da9b: Generated new TSK 0
I20260925 11:00:53.008630 28367 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:00:53.054551 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44535
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:46189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:53.366475 28381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:53.366806 28381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:53.366937 28381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:53.374322 28381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:53.374456 28381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:53.374611 28381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:00:53.384583 28381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:46189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:53.385962 28381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:53.387375 28381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:53.399997 28389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:53.399960 28387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:53.399927 28386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:53.400619 28381 server_base.cc:1034] running on GCE node
I20260925 11:00:53.401239 28381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:53.402238 28381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:53.403455 28381 hybrid_clock.cc:648] HybridClock initialized: now 1790334053403386 us; error 66 us; skew 500 ppm
I20260925 11:00:53.403767 28381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:53 dist-test-slave-trms krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334053, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:53.406813 28381 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:00:53.408200 28381 webserver.cc:460] Webserver started at http://127.24.159.193:34993/ using document root <none> and password file <none>
I20260925 11:00:53.408915 28381 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:53.409030 28381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:53.409313 28381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:53.412279 28381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "38d8e3af680d4e07bdbcb16e84c73951"
format_stamp: "Formatted at 2026-09-25 11:00:53 on dist-test-slave-trms"
server_key: "9c0b5f9c9b1d391ff7f8a474ed068f1b"
server_key_iv: "b2f184a565a610d826ebd6594876eb4b"
server_key_version: "encryptionkey@0"
I20260925 11:00:53.413043 28381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "38d8e3af680d4e07bdbcb16e84c73951"
format_stamp: "Formatted at 2026-09-25 11:00:53 on dist-test-slave-trms"
server_key: "9c0b5f9c9b1d391ff7f8a474ed068f1b"
server_key_iv: "b2f184a565a610d826ebd6594876eb4b"
server_key_version: "encryptionkey@0"
I20260925 11:00:53.418120 28381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20260925 11:00:53.421563 28396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:53.422982 28381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:53.423130 28381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "38d8e3af680d4e07bdbcb16e84c73951"
format_stamp: "Formatted at 2026-09-25 11:00:53 on dist-test-slave-trms"
server_key: "9c0b5f9c9b1d391ff7f8a474ed068f1b"
server_key_iv: "b2f184a565a610d826ebd6594876eb4b"
server_key_version: "encryptionkey@0"
I20260925 11:00:53.423326 28381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:53.439731 28381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:53.443838 28381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:53.444103 28381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:53.444977 28381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:53.446329 28381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:53.446405 28381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:53.446529 28381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:53.446580 28381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:53.476123 28381 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:37685
I20260925 11:00:53.476163 28508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:37685 every 8 connection(s)
I20260925 11:00:53.477403 28381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:00:53.483012 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 28381
I20260925 11:00:53.483222 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:00:53.484009 25215 external_mini_cluster.cc:1428] Setting key b62175b6b1371335ddd28e5ec72ca531
Sep 25 11:00:53 dist-test-slave-trms krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334053, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:53.498231 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:53.479888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:38887 (local address 127.24.159.254:44535)
0925 11:00:53.480318 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:53.480323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:53.481246 (+ 923us) server_negotiation.cc:407] Connection header received
0925 11:00:53.483244 (+ 1998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:53.483256 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:53.483346 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:53.483449 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:53.485598 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:53.486677 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:53.488645 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:53.489192 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:53.489315 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:53.492750 (+ 3435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:53.492754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:53.492758 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:53.492788 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:53.495204 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:53.495902 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:53.495907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:53.495911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:53.496042 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:53.496443 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:53.496448 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:53.496452 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:53.496730 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:53.496858 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:53.497614 (+ 756us) server_negotiation.cc:299] Negotiation successful
0925 11:00:53.497840 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":224,"threads_started":1}
I20260925 11:00:53.499202 28512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:53.480410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44535 (local address 127.24.159.193:38887)
0925 11:00:53.481093 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:53.481137 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:53.482888 (+ 1751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:53.483749 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:53.483762 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:53.484523 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:53.485380 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:53.485407 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:53.486873 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:53.486879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:53.488499 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:53.488512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:53.489386 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:53.489391 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:53.489584 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:53.490229 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:53.490266 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:53.492547 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:53.495338 (+ 2791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:53.495348 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:53.495366 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:53.495783 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:53.496145 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:53.496150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:53.496154 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:53.496341 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:53.496850 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:53.496858 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:53.497234 (+ 376us) client_negotiation.cc:769] Sending connection context
0925 11:00:53.497522 (+ 288us) client_negotiation.cc:240] Negotiation successful
0925 11:00:53.498075 (+ 553us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":251,"threads_started":1}
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
I20260925 11:00:53.500703 28509 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44535
I20260925 11:00:53.500998 28509 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:53.501760 28509 heartbeater.cc:511] Master 127.24.159.254:44535 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
I20260925 11:00:53.503819 28320 ts_manager.cc:194] Registered new tserver with Master: 38d8e3af680d4e07bdbcb16e84c73951 (127.24.159.193:37685)
I20260925 11:00:53.505283 28320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:38887
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:00:53.523334 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44535
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:46189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:53.828807 28515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:53.829087 28515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:53.829205 28515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:53.836211 28515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:53.836323 28515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:53.836472 28515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:00:53.845798 28515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:46189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:53.847064 28515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:53.848400 28515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:53.860721 28520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:53.860643 28521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:53.860673 28523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:53.861366 28515 server_base.cc:1034] running on GCE node
I20260925 11:00:53.861948 28515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:53.862933 28515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:53.864127 28515 hybrid_clock.cc:648] HybridClock initialized: now 1790334053864072 us; error 50 us; skew 500 ppm
I20260925 11:00:53.864442 28515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:53 dist-test-slave-trms krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334053, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:53.867317 28515 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:00:53.868757 28515 webserver.cc:460] Webserver started at http://127.24.159.194:41739/ using document root <none> and password file <none>
I20260925 11:00:53.869391 28515 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:53.869513 28515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:53.869782 28515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:53.872611 28515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "b53a336a23144d1488358f5f2b4596f2"
format_stamp: "Formatted at 2026-09-25 11:00:53 on dist-test-slave-trms"
server_key: "5d5e644d8fa44ce6a41ff8ffd363f95e"
server_key_iv: "22060a89c820c5a3a95b9dcd37edba95"
server_key_version: "encryptionkey@0"
I20260925 11:00:53.873344 28515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "b53a336a23144d1488358f5f2b4596f2"
format_stamp: "Formatted at 2026-09-25 11:00:53 on dist-test-slave-trms"
server_key: "5d5e644d8fa44ce6a41ff8ffd363f95e"
server_key_iv: "22060a89c820c5a3a95b9dcd37edba95"
server_key_version: "encryptionkey@0"
I20260925 11:00:53.878178 28515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 11:00:53.881521 28530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:53.882835 28515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:53.882985 28515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "b53a336a23144d1488358f5f2b4596f2"
format_stamp: "Formatted at 2026-09-25 11:00:53 on dist-test-slave-trms"
server_key: "5d5e644d8fa44ce6a41ff8ffd363f95e"
server_key_iv: "22060a89c820c5a3a95b9dcd37edba95"
server_key_version: "encryptionkey@0"
I20260925 11:00:53.883157 28515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:53.905411 28515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:53.909274 28515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:53.909557 28515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:53.910621 28515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:53.911934 28515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:53.912016 28515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:53.912132 28515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:53.912184 28515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:53.941705 28515 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:43947
I20260925 11:00:53.941743 28642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:43947 every 8 connection(s)
I20260925 11:00:53.943011 28515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:00:53.949595 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 28515
I20260925 11:00:53.949745 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:00:53.950248 25215 external_mini_cluster.cc:1428] Setting key 77744e67a58e66cc8e35d2d5f949d374
Sep 25 11:00:53 dist-test-slave-trms krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334053, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:53.962860 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:53.945382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:54519 (local address 127.24.159.254:44535)
0925 11:00:53.945547 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:53.945552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:53.946726 (+ 1174us) server_negotiation.cc:407] Connection header received
0925 11:00:53.948439 (+ 1713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:53.948444 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:53.948511 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:53.948595 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:53.950078 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:53.951045 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:53.952667 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:53.953257 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:53.953396 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:53.956986 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:53.956992 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:53.956995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:53.957052 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:53.960042 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:53.960733 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:53.960738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:53.960742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:53.960889 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:53.961244 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:53.961248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:53.961251 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:53.961565 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:53.961715 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:53.962334 (+ 619us) server_negotiation.cc:299] Negotiation successful
0925 11:00:53.962556 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 11:00:53.963892 28645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:53.945846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44535 (local address 127.24.159.194:54519)
0925 11:00:53.946568 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:53.946612 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:53.948201 (+ 1589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:53.948802 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:53.948810 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:53.949298 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:53.949898 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:53.949928 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:53.951183 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:53.951187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:53.952521 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:53.952533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:53.953446 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:53.953451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:53.953643 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:53.954316 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:53.954338 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:53.956760 (+ 2422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:53.960186 (+ 3426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:53.960193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:53.960207 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:53.960629 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:53.960982 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:53.960986 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:53.960988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:53.961157 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:53.961666 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:53.961672 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:53.961988 (+ 316us) client_negotiation.cc:769] Sending connection context
0925 11:00:53.962236 (+ 248us) client_negotiation.cc:240] Negotiation successful
0925 11:00:53.962657 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":281,"threads_started":1}
I20260925 11:00:53.965814 28643 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44535
I20260925 11:00:53.966166 28643 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:53.967085 28643 heartbeater.cc:511] Master 127.24.159.254:44535 requested a full tablet report, sending...
I20260925 11:00:53.968596 28320 ts_manager.cc:194] Registered new tserver with Master: b53a336a23144d1488358f5f2b4596f2 (127.24.159.194:43947)
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
I20260925 11:00:53.969389 28320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:54519
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:00:53.992785 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44535
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:46189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:54.307264 28648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:54.307615 28648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:54.307763 28648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:54.314945 28648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:54.315065 28648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:54.315213 28648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:00:54.324723 28648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:46189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:54.325965 28648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:54.327358 28648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:54.340108 28654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:54.340130 28653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:54.340071 28656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:54.340895 28648 server_base.cc:1034] running on GCE node
I20260925 11:00:54.341434 28648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:54.342311 28648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:54.343532 28648 hybrid_clock.cc:648] HybridClock initialized: now 1790334054343484 us; error 34 us; skew 500 ppm
I20260925 11:00:54.343830 28648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:54.346747 28648 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:00:54.348153 28648 webserver.cc:460] Webserver started at http://127.24.159.195:37317/ using document root <none> and password file <none>
I20260925 11:00:54.348809 28648 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:54.348928 28648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:54.349202 28648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:54.351969 28648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "25f2e65493ed4c689beacf26badb434f"
format_stamp: "Formatted at 2026-09-25 11:00:54 on dist-test-slave-trms"
server_key: "18064ed752c9d04b37631a97a84ebb8b"
server_key_iv: "27cbe6b2848fbcfc86e7da0444781786"
server_key_version: "encryptionkey@0"
I20260925 11:00:54.352684 28648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "25f2e65493ed4c689beacf26badb434f"
format_stamp: "Formatted at 2026-09-25 11:00:54 on dist-test-slave-trms"
server_key: "18064ed752c9d04b37631a97a84ebb8b"
server_key_iv: "27cbe6b2848fbcfc86e7da0444781786"
server_key_version: "encryptionkey@0"
I20260925 11:00:54.357558 28648 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 11:00:54.360832 28663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:54.362145 28648 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:00:54.362318 28648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "25f2e65493ed4c689beacf26badb434f"
format_stamp: "Formatted at 2026-09-25 11:00:54 on dist-test-slave-trms"
server_key: "18064ed752c9d04b37631a97a84ebb8b"
server_key_iv: "27cbe6b2848fbcfc86e7da0444781786"
server_key_version: "encryptionkey@0"
I20260925 11:00:54.362489 28648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:54.380560 28648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:54.384377 28648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:54.384634 28648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:54.385546 28648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:54.386898 28648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:54.386981 28648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:54.387097 28648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:54.387146 28648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:54.416124 28648 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:43431
I20260925 11:00:54.416162 28775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:43431 every 8 connection(s)
I20260925 11:00:54.417352 28648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:00:54.419504 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 28648
I20260925 11:00:54.419672 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:00:54.420215 25215 external_mini_cluster.cc:1428] Setting key 322c64fd78e3fa611d4930bd826491a1
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:54.436339 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:54.419974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:54113 (local address 127.24.159.254:44535)
0925 11:00:54.420129 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:54.420134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:54.421624 (+ 1490us) server_negotiation.cc:407] Connection header received
0925 11:00:54.423398 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:54.423402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:54.423468 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:54.423543 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:54.424960 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.425675 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.427319 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.427763 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.427892 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:54.430995 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:54.431000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:54.431003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:54.431035 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:54.433590 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.434319 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.434324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.434327 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.434477 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.434872 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.434876 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.434879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.435190 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:54.435330 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:54.435918 (+ 588us) server_negotiation.cc:299] Negotiation successful
0925 11:00:54.436108 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 11:00:54.437239 28778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:54.420678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44535 (local address 127.24.159.195:54113)
0925 11:00:54.421465 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:54.421510 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:54.423161 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:54.423741 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:54.423748 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:54.424234 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:54.424809 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:54.424826 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.425797 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.425800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:54.427198 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:54.427210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.427956 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.427961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:54.428151 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:54.428712 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:54.428739 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:54.430779 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:54.433728 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:54.433735 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:54.433747 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:54.434210 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:54.434581 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:54.434584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:54.434587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:54.434764 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:54.435312 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:54.435318 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:54.435628 (+ 310us) client_negotiation.cc:769] Sending connection context
0925 11:00:54.435844 (+ 216us) client_negotiation.cc:240] Negotiation successful
0925 11:00:54.436219 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":307,"threads_started":1}
I20260925 11:00:54.438422 28776 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44535
I20260925 11:00:54.438699 28776 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:54.439395 28776 heartbeater.cc:511] Master 127.24.159.254:44535 requested a full tablet report, sending...
I20260925 11:00:54.440752 28320 ts_manager.cc:194] Registered new tserver with Master: 25f2e65493ed4c689beacf26badb434f (127.24.159.195:43431)
I20260925 11:00:54.441486 28320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:54113
I20260925 11:00:54.447359 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:54.493503 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:54.479051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57118 (local address 127.24.159.254:44535)
0925 11:00:54.479318 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:54.479323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:54.479724 (+ 401us) server_negotiation.cc:407] Connection header received
0925 11:00:54.479900 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:54.479905 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:54.479974 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:54.480072 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:54.481429 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.482154 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.483417 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.483839 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.483950 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:54.487686 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:54.487691 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:54.487695 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:54.487730 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:54.490919 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.491527 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.491534 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.491538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.491673 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.492025 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.492031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.492034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.492358 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:54.492587 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:54.492891 (+ 304us) server_negotiation.cc:299] Negotiation successful
0925 11:00:54.493097 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260925 11:00:54.494092 28320 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57118
I20260925 11:00:54.495738 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28381
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:00:54.508814 28509 heartbeater.cc:503] Master 127.24.159.254:44535 was elected leader, sending a full tablet report...
I20260925 11:00:54.511140 28787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:54.496702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.24.159.193:37685)
0925 11:00:54.497268 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:54.497275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:54.497302 (+ 27us) server_negotiation.cc:407] Connection header received
0925 11:00:54.497364 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:54.497370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:54.497575 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:54.497707 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:54.498849 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.499904 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.501020 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.501573 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.501670 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:54.505388 (+ 3718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:54.505404 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:54.505421 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:54.505457 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:54.508366 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.509182 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.509190 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.509192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.509325 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.509707 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.509712 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.509714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.510061 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:54.510350 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:54.510651 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 11:00:54.510865 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":295,"threads_started":1}
W20260925 11:00:54.512318 28504 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37172
W20260925 11:00:54.513383 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28381: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:54.513585 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 28381
I20260925 11:00:54.525887 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28515
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:00:54.541136 28788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:54.526840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51562 (local address 127.24.159.194:43947)
0925 11:00:54.527376 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:54.527384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:54.527409 (+ 25us) server_negotiation.cc:407] Connection header received
0925 11:00:54.527470 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:54.527476 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:54.527687 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:54.527813 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:54.528946 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.529912 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.531131 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.531674 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.531773 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:54.535819 (+ 4046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:54.535830 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:54.535845 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:54.535890 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:54.538680 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.539219 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.539225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.539227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.539351 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.539664 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.539667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.539669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.539997 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:54.540212 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:54.540556 (+ 344us) server_negotiation.cc:299] Negotiation successful
0925 11:00:54.540813 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":254,"threads_started":1}
W20260925 11:00:54.542500 28638 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:51562
W20260925 11:00:54.543566 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28515: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:54.543700 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 28515
I20260925 11:00:54.556066 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28648
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334054, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:00:54.571578 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:54.557106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.24.159.195:43431)
0925 11:00:54.557782 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:54.557789 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:54.557818 (+ 29us) server_negotiation.cc:407] Connection header received
0925 11:00:54.557879 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:54.557885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:54.558113 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:54.558300 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:54.559491 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.560549 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.561678 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:54.562275 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:54.562380 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:54.566387 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:54.566400 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:54.566414 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:54.566449 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:54.569081 (+ 2632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.569682 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.569689 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.569692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.569833 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:54.570109 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:54.570112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:54.570114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:54.570496 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:54.570712 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:54.570998 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 11:00:54.571225 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":355,"threads_started":1}
W20260925 11:00:54.572796 28771 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35148
W20260925 11:00:54.573850 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28648: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:00:54.573999 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 28648
I20260925 11:00:54.586802 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28288
I20260925 11:00:54.587978 28350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:54.691303 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 28288
2026-09-25T11:00:54Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3354 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28793](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28793](info): set up 2 sockets
Sep 25 11:00:54 dist-test-slave-trms krb5kdc[28793](info): commencing operation
krb5kdc: starting...
W20260925 11:00:55.702589 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:00:55 dist-test-slave-trms krb5kdc[28793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:00:55.882295 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45817
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:45817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:56.193539 28806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:56.193833 28806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:56.193972 28806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:56.201140 28806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:56.201247 28806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:56.201323 28806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:56.201399 28806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:56.201463 28806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:56.211133 28806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:45817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:56.212426 28806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:56.213660 28806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:56.224591 28811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:56.224658 28812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:56.224622 28814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:56.225186 28806 server_base.cc:1034] running on GCE node
I20260925 11:00:56.226018 28806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:56.227358 28806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:56.228567 28806 hybrid_clock.cc:648] HybridClock initialized: now 1790334056228499 us; error 57 us; skew 500 ppm
I20260925 11:00:56.228889 28806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:56 dist-test-slave-trms krb5kdc[28793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334056, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:56.234537 28806 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:00:56.235935 28806 webserver.cc:460] Webserver started at http://127.24.159.254:35237/ using document root <none> and password file <none>
I20260925 11:00:56.236595 28806 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:56.236706 28806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:56.236984 28806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:56.239871 28806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "9cb12d533af5497ca2c049496ee475ce"
format_stamp: "Formatted at 2026-09-25 11:00:56 on dist-test-slave-trms"
server_key: "fa2df89bc007ab1dfa8f72489691d278"
server_key_iv: "54030941b63477927560e9fde60a6c64"
server_key_version: "encryptionkey@0"
I20260925 11:00:56.240624 28806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "9cb12d533af5497ca2c049496ee475ce"
format_stamp: "Formatted at 2026-09-25 11:00:56 on dist-test-slave-trms"
server_key: "fa2df89bc007ab1dfa8f72489691d278"
server_key_iv: "54030941b63477927560e9fde60a6c64"
server_key_version: "encryptionkey@0"
I20260925 11:00:56.245612 28806 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 11:00:56.249080 28821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:56.250574 28806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:00:56.250751 28806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "9cb12d533af5497ca2c049496ee475ce"
format_stamp: "Formatted at 2026-09-25 11:00:56 on dist-test-slave-trms"
server_key: "fa2df89bc007ab1dfa8f72489691d278"
server_key_iv: "54030941b63477927560e9fde60a6c64"
server_key_version: "encryptionkey@0"
I20260925 11:00:56.250928 28806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:56.271144 28806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:56.275182 28806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:56.275452 28806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:56.292040 28806 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:45817
I20260925 11:00:56.292058 28872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:45817 every 8 connection(s)
I20260925 11:00:56.293375 28806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:56.297006 28873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:56.299100 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 28806
I20260925 11:00:56.299328 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:00:56.300067 25215 external_mini_cluster.cc:1428] Setting key d007d2b1ea2d8137d0a55862bcbbf852
I20260925 11:00:56.309262 28873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce: Bootstrap starting.
I20260925 11:00:56.313015 28873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:56.314263 28873 log.cc:824] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce: Log is configured to *not* fsync() on all Append() calls
Sep 25 11:00:56 dist-test-slave-trms krb5kdc[28793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:56.317713 28873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce: No bootstrap required, opened a new log
I20260925 11:00:56.323721 28876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:56.302126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.24.159.254:45817)
0925 11:00:56.302931 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:56.302943 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:56.302987 (+ 44us) server_negotiation.cc:407] Connection header received
0925 11:00:56.304608 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:56.304631 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:56.305034 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:56.305369 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:56.307057 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:56.308356 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:56.310062 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:56.310841 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:56.310988 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:56.315567 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:56.315577 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:56.315598 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:56.315643 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:56.319225 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:56.319856 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:56.319863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:56.319872 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:56.320088 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:56.320430 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:56.320435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:56.320441 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:56.321189 (+ 748us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:56.321491 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:56.321805 (+ 314us) server_negotiation.cc:299] Negotiation successful
0925 11:00:56.322299 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":341,"threads_started":1}
I20260925 11:00:56.327042 28873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb12d533af5497ca2c049496ee475ce" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45817 } }
I20260925 11:00:56.327363 28873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:56.327440 28873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb12d533af5497ca2c049496ee475ce, State: Initialized, Role: FOLLOWER
I20260925 11:00:56.327963 28873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb12d533af5497ca2c049496ee475ce" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45817 } }
I20260925 11:00:56.328166 28873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:56.328262 28873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:56.328383 28873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:56.330510 28873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb12d533af5497ca2c049496ee475ce" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45817 } }
I20260925 11:00:56.331000 28873 leader_election.cc:302] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cb12d533af5497ca2c049496ee475ce; no voters:
I20260925 11:00:56.331539 28873 leader_election.cc:288] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:56.331660 28878 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:56.332304 28878 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [term 1 LEADER]: Becoming Leader. State: Replica: 9cb12d533af5497ca2c049496ee475ce, State: Running, Role: LEADER
I20260925 11:00:56.332803 28878 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb12d533af5497ca2c049496ee475ce" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45817 } }
I20260925 11:00:56.333446 28873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:56.335515 28880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cb12d533af5497ca2c049496ee475ce. Latest consensus state: current_term: 1 leader_uuid: "9cb12d533af5497ca2c049496ee475ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb12d533af5497ca2c049496ee475ce" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45817 } } }
I20260925 11:00:56.335889 28880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:56.336608 28879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cb12d533af5497ca2c049496ee475ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cb12d533af5497ca2c049496ee475ce" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45817 } } }
I20260925 11:00:56.336889 28879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:56.338443 28883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:56.343348 28883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:56.352591 28883 catalog_manager.cc:1347] Generated new cluster ID: c37961aae77b4c4688afe4e26c07c3e6
I20260925 11:00:56.352700 28883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:56.387526 28883 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:00:56.388914 28883 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:00:56.396008 28883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cb12d533af5497ca2c049496ee475ce: Generated new TSK 0
I20260925 11:00:56.396836 28883 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:00:56.449383 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45817
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:56.765152 28899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:56.765439 28899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:56.765532 28899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:56.772711 28899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:56.772830 28899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:56.772948 28899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:00:56.782792 28899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:56.784029 28899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:56.785331 28899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:56.797796 28904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:56.797755 28905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:56.797824 28907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:56.798564 28899 server_base.cc:1034] running on GCE node
I20260925 11:00:56.799119 28899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:56.799996 28899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:56.801190 28899 hybrid_clock.cc:648] HybridClock initialized: now 1790334056801141 us; error 44 us; skew 500 ppm
I20260925 11:00:56.801468 28899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:56 dist-test-slave-trms krb5kdc[28793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334056, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:56.804425 28899 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:00:56.805778 28899 webserver.cc:460] Webserver started at http://127.24.159.193:39259/ using document root <none> and password file <none>
I20260925 11:00:56.806493 28899 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:56.806594 28899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:56.806838 28899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:56.809705 28899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "1ece94e9193744baade99c0c20c2162e"
format_stamp: "Formatted at 2026-09-25 11:00:56 on dist-test-slave-trms"
server_key: "08e5626704700da4ad366121285ceb96"
server_key_iv: "5b4f7035be58009083ec4dc939b7739c"
server_key_version: "encryptionkey@0"
I20260925 11:00:56.810477 28899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "1ece94e9193744baade99c0c20c2162e"
format_stamp: "Formatted at 2026-09-25 11:00:56 on dist-test-slave-trms"
server_key: "08e5626704700da4ad366121285ceb96"
server_key_iv: "5b4f7035be58009083ec4dc939b7739c"
server_key_version: "encryptionkey@0"
I20260925 11:00:56.815732 28899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 11:00:56.819007 28914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:56.820318 28899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:00:56.820441 28899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "1ece94e9193744baade99c0c20c2162e"
format_stamp: "Formatted at 2026-09-25 11:00:56 on dist-test-slave-trms"
server_key: "08e5626704700da4ad366121285ceb96"
server_key_iv: "5b4f7035be58009083ec4dc939b7739c"
server_key_version: "encryptionkey@0"
I20260925 11:00:56.820608 28899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:56.836400 28899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:56.840487 28899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:56.840741 28899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:56.841656 28899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:56.842978 28899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:56.843051 28899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:56.843171 28899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:56.843225 28899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:56.870352 28899 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:36975
I20260925 11:00:56.870393 29026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:36975 every 8 connection(s)
I20260925 11:00:56.871591 28899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:00:56.876082 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 28899
I20260925 11:00:56.876266 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:00:56.876839 25215 external_mini_cluster.cc:1428] Setting key 22cf484d2e5a278e871c4b0b0276c1bc
Sep 25 11:00:56 dist-test-slave-trms krb5kdc[28793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334056, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:56.891345 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:56.874034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:56377 (local address 127.24.159.254:45817)
0925 11:00:56.874490 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:56.874499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:56.875438 (+ 939us) server_negotiation.cc:407] Connection header received
0925 11:00:56.877286 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:56.877292 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:56.877372 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:56.877473 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:56.878976 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:56.879689 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:56.881192 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:56.881596 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:56.881702 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:56.885335 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:56.885339 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:56.885343 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:56.885376 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:56.887881 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:56.888808 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:56.888812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:56.888814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:56.888927 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:56.889371 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:56.889375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:56.889378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:56.889725 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:56.889841 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:56.890896 (+ 1055us) server_negotiation.cc:299] Negotiation successful
0925 11:00:56.891079 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":241,"threads_started":1}
I20260925 11:00:56.892794 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:56.874585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:45817 (local address 127.24.159.193:56377)
0925 11:00:56.875279 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:56.875317 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:56.877029 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:56.877706 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:56.877715 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:56.878249 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:56.878819 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:56.878835 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:56.879837 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:56.879841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:56.881060 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:56.881072 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:56.881810 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:56.881817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:56.882053 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:56.882694 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:56.882718 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:56.885091 (+ 2373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:56.888032 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:56.888041 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:56.888061 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:56.888700 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:56.889017 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:56.889021 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:56.889024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:56.889276 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:56.889854 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:56.889863 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:56.890399 (+ 536us) client_negotiation.cc:769] Sending connection context
0925 11:00:56.890730 (+ 331us) client_negotiation.cc:240] Negotiation successful
0925 11:00:56.891312 (+ 582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":274,"threads_started":1}
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
I20260925 11:00:56.894443 29027 heartbeater.cc:347] Connected to a master server at 127.24.159.254:45817
I20260925 11:00:56.894768 29027 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:56.895570 29027 heartbeater.cc:511] Master 127.24.159.254:45817 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
I20260925 11:00:56.897815 28838 ts_manager.cc:194] Registered new tserver with Master: 1ece94e9193744baade99c0c20c2162e (127.24.159.193:36975)
I20260925 11:00:56.899329 28838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:56377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:00:56.917925 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45817
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:57.230091 29033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:57.230407 29033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:57.230564 29033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:57.237807 29033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:57.237910 29033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:57.238046 29033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:00:57.247732 29033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:57.249010 29033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:57.250300 29033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:57.262722 29041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:57.262809 29038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:57.262789 29039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:57.263581 29033 server_base.cc:1034] running on GCE node
I20260925 11:00:57.264128 29033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:57.265004 29033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:57.266235 29033 hybrid_clock.cc:648] HybridClock initialized: now 1790334057266163 us; error 46 us; skew 500 ppm
I20260925 11:00:57.266535 29033 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:57 dist-test-slave-trms krb5kdc[28793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334057, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:57.269433 29033 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:00:57.270879 29033 webserver.cc:460] Webserver started at http://127.24.159.194:42871/ using document root <none> and password file <none>
I20260925 11:00:57.271529 29033 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:57.271641 29033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:57.271926 29033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:57.274732 29033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "19734dc192f44fed92c865aad996664e"
format_stamp: "Formatted at 2026-09-25 11:00:57 on dist-test-slave-trms"
server_key: "1f1115f4770edcfeed1b57d152dce38a"
server_key_iv: "8b5967801e66af4a36aa291456f2a302"
server_key_version: "encryptionkey@0"
I20260925 11:00:57.275489 29033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "19734dc192f44fed92c865aad996664e"
format_stamp: "Formatted at 2026-09-25 11:00:57 on dist-test-slave-trms"
server_key: "1f1115f4770edcfeed1b57d152dce38a"
server_key_iv: "8b5967801e66af4a36aa291456f2a302"
server_key_version: "encryptionkey@0"
I20260925 11:00:57.280592 29033 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:00:57.284228 29048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:57.285550 29033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260925 11:00:57.285703 29033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "19734dc192f44fed92c865aad996664e"
format_stamp: "Formatted at 2026-09-25 11:00:57 on dist-test-slave-trms"
server_key: "1f1115f4770edcfeed1b57d152dce38a"
server_key_iv: "8b5967801e66af4a36aa291456f2a302"
server_key_version: "encryptionkey@0"
I20260925 11:00:57.285902 29033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:57.313627 29033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:57.317881 29033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:57.318137 29033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:57.319049 29033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:57.320322 29033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:57.320391 29033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:57.320498 29033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:57.320554 29033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:57.348090 29033 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:46605
I20260925 11:00:57.348127 29160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:46605 every 8 connection(s)
I20260925 11:00:57.349328 29033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:00:57.355443 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 29033
I20260925 11:00:57.355641 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:00:57.356032 25215 external_mini_cluster.cc:1428] Setting key 353b3fde5d24f6d4c7317dfb78f6c9a0
Sep 25 11:00:57 dist-test-slave-trms krb5kdc[28793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334057, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:57.369784 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:57.351670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:35993 (local address 127.24.159.254:45817)
0925 11:00:57.351829 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:57.351833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:57.353009 (+ 1176us) server_negotiation.cc:407] Connection header received
0925 11:00:57.354877 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:57.354883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:57.354967 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:57.355051 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:57.356687 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.357451 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.359643 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.360044 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.360152 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:57.363938 (+ 3786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:57.363944 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:57.363948 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:57.363980 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:57.366890 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.367609 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.367614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.367617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.367768 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.368151 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.368156 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.368159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.368470 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:57.368604 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:57.369270 (+ 666us) server_negotiation.cc:299] Negotiation successful
0925 11:00:57.369482 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 11:00:57.370728 29163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:57.352172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:45817 (local address 127.24.159.194:35993)
0925 11:00:57.352844 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:57.352892 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:57.354607 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:57.355314 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:57.355323 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:57.355880 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:57.356498 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:57.356524 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.357600 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.357606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:57.359501 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:57.359517 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.360258 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.360264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:57.360545 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:57.361217 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:57.361244 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:57.363700 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:57.367045 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:57.367052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:57.367067 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:57.367487 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:57.367864 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:57.367867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:57.367869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:57.368046 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:57.368569 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:57.368574 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:57.368908 (+ 334us) client_negotiation.cc:769] Sending connection context
0925 11:00:57.369128 (+ 220us) client_negotiation.cc:240] Negotiation successful
0925 11:00:57.369527 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":274,"threads_started":1}
I20260925 11:00:57.372160 29161 heartbeater.cc:347] Connected to a master server at 127.24.159.254:45817
I20260925 11:00:57.372449 29161 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:57.373147 29161 heartbeater.cc:511] Master 127.24.159.254:45817 requested a full tablet report, sending...
I20260925 11:00:57.374620 28838 ts_manager.cc:194] Registered new tserver with Master: 19734dc192f44fed92c865aad996664e (127.24.159.194:46605)
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
I20260925 11:00:57.375576 28838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:35993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:00:57.397799 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45817
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:57.708118 29166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:57.708380 29166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:57.708479 29166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:57.715363 29166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:57.715469 29166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:57.715585 29166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:00:57.724980 29166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45817
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:57.726258 29166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:57.727494 29166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:57.739746 29171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:57.739785 29172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:57.739816 29174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:57.740610 29166 server_base.cc:1034] running on GCE node
I20260925 11:00:57.741138 29166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:57.742000 29166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:57.743202 29166 hybrid_clock.cc:648] HybridClock initialized: now 1790334057743140 us; error 48 us; skew 500 ppm
I20260925 11:00:57.743494 29166 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:57 dist-test-slave-trms krb5kdc[28793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334057, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:57.746373 29166 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:00:57.747720 29166 webserver.cc:460] Webserver started at http://127.24.159.195:33033/ using document root <none> and password file <none>
I20260925 11:00:57.748342 29166 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:57.748435 29166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:57.748675 29166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:57.751439 29166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "02ef736be0da41e5aed64bac72f49cab"
format_stamp: "Formatted at 2026-09-25 11:00:57 on dist-test-slave-trms"
server_key: "15a9b0b4186ca00990d4f0ac65c5ff5b"
server_key_iv: "c38fa147629020adaddb92f3b5a27dbd"
server_key_version: "encryptionkey@0"
I20260925 11:00:57.752168 29166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "02ef736be0da41e5aed64bac72f49cab"
format_stamp: "Formatted at 2026-09-25 11:00:57 on dist-test-slave-trms"
server_key: "15a9b0b4186ca00990d4f0ac65c5ff5b"
server_key_iv: "c38fa147629020adaddb92f3b5a27dbd"
server_key_version: "encryptionkey@0"
I20260925 11:00:57.757071 29166 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:00:57.760289 29181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:57.761497 29166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260925 11:00:57.761626 29166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "02ef736be0da41e5aed64bac72f49cab"
format_stamp: "Formatted at 2026-09-25 11:00:57 on dist-test-slave-trms"
server_key: "15a9b0b4186ca00990d4f0ac65c5ff5b"
server_key_iv: "c38fa147629020adaddb92f3b5a27dbd"
server_key_version: "encryptionkey@0"
I20260925 11:00:57.761796 29166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:57.785302 29166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:57.789620 29166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:57.789860 29166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:57.790822 29166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:00:57.792068 29166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:00:57.792140 29166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:57.792248 29166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:00:57.792296 29166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:57.819778 29166 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:34957
I20260925 11:00:57.819823 29293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:34957 every 8 connection(s)
I20260925 11:00:57.821002 29166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:00:57.823326 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 29166
I20260925 11:00:57.823496 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:00:57.823949 25215 external_mini_cluster.cc:1428] Setting key 3f839a9e32468a23bafeda864fefd571
Sep 25 11:00:57 dist-test-slave-trms krb5kdc[28793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334057, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:57.840353 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:57.823849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:33045 (local address 127.24.159.254:45817)
0925 11:00:57.824077 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:57.824084 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:57.825485 (+ 1401us) server_negotiation.cc:407] Connection header received
0925 11:00:57.827572 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:57.827576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:57.827645 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:57.827723 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:57.829154 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.829876 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.831408 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.831834 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.831935 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:57.835127 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:57.835132 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:57.835135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:57.835168 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:57.837397 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.838234 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.838240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.838243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.838361 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.838801 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.838805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.838808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.839158 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:57.839269 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:57.839926 (+ 657us) server_negotiation.cc:299] Negotiation successful
0925 11:00:57.840106 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:00:57.841353 29296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:57.824476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:45817 (local address 127.24.159.195:33045)
0925 11:00:57.825317 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0925 11:00:57.825377 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 11:00:57.827333 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:00:57.827903 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:00:57.827911 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:00:57.828414 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:00:57.829017 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:57.829034 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.829998 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.830001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:57.831276 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:00:57.831288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.832009 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.832013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:00:57.832202 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:57.832776 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:00:57.832798 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:00:57.834926 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:00:57.837536 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:57.837545 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:57.837564 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:57.838093 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:57.838472 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:00:57.838478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:00:57.838482 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:00:57.838690 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:00:57.839280 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:00:57.839287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:00:57.839614 (+ 327us) client_negotiation.cc:769] Sending connection context
0925 11:00:57.839844 (+ 230us) client_negotiation.cc:240] Negotiation successful
0925 11:00:57.840224 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":360,"threads_started":1}
I20260925 11:00:57.842700 29294 heartbeater.cc:347] Connected to a master server at 127.24.159.254:45817
I20260925 11:00:57.842979 29294 heartbeater.cc:464] Registering TS with master...
I20260925 11:00:57.843631 29294 heartbeater.cc:511] Master 127.24.159.254:45817 requested a full tablet report, sending...
I20260925 11:00:57.844962 28838 ts_manager.cc:194] Registered new tserver with Master: 02ef736be0da41e5aed64bac72f49cab (127.24.159.195:34957)
I20260925 11:00:57.845752 28838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:33045
I20260925 11:00:57.851150 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 25 11:00:57 dist-test-slave-trms krb5kdc[28793](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 11:00:57 dist-test-slave-trms krb5kdc[28793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:00:57.894793 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:57.880499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50428 (local address 127.24.159.193:36975)
0925 11:00:57.881054 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:57.881064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:57.881098 (+ 34us) server_negotiation.cc:407] Connection header received
0925 11:00:57.881288 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:57.881296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:57.881512 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:57.881649 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:57.883079 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.884077 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.885299 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.885851 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.885973 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:57.889568 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:57.889579 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:57.889593 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:57.889631 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:57.892146 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.892762 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.892769 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.892772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.892893 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.893247 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.893251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.893253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.893588 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:57.893868 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:57.894155 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 11:00:57.894453 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":240,"threads_started":1}
I20260925 11:00:57.902292 29027 heartbeater.cc:503] Master 127.24.159.254:45817 was elected leader, sending a full tablet report...
Sep 25 11:00:57 dist-test-slave-trms krb5kdc[28793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:57.911626 29029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:57.898722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41346 (local address 127.24.159.254:45817)
0925 11:00:57.898956 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:57.898962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:57.899137 (+ 175us) server_negotiation.cc:407] Connection header received
0925 11:00:57.899310 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:57.899314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:57.899380 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:57.899462 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:57.900642 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.901359 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.902631 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.903033 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.903138 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:57.906645 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:57.906650 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:57.906653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:57.906690 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:57.909160 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.909765 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.909769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.909771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.909898 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.910265 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.910268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.910270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.910584 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:57.910807 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:57.911123 (+ 316us) server_negotiation.cc:299] Negotiation successful
0925 11:00:57.911347 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260925 11:00:57.912129 28838 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41346
I20260925 11:00:57.913050 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28899
I20260925 11:00:57.925498 29301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:57.913919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50444 (local address 127.24.159.193:36975)
0925 11:00:57.914110 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:57.914116 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:57.914265 (+ 149us) server_negotiation.cc:407] Connection header received
0925 11:00:57.914420 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:57.914426 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:57.914487 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:57.914565 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:57.915700 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.916380 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.917529 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:57.917933 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:57.918037 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:57.920137 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:57.920142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:57.920145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:57.920177 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:57.922474 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.923162 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.923168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.923187 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.923361 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:57.923763 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:57.923768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:57.923771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:57.924130 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:57.924381 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:57.924922 (+ 541us) server_negotiation.cc:299] Negotiation successful
0925 11:00:57.925150 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:00:57.926272 29022 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:58.010788 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 28899
I20260925 11:00:58.023291 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 29033
Sep 25 11:00:58 dist-test-slave-trms krb5kdc[28793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:00:58.041505 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:58.024143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41030 (local address 127.24.159.194:46605)
0925 11:00:58.024617 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:58.024624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:58.024650 (+ 26us) server_negotiation.cc:407] Connection header received
0925 11:00:58.024708 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:58.024714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:58.024929 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:58.025052 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:58.026114 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:58.027697 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:58.028998 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:58.029824 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:58.030532 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:58.035007 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:58.035021 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:58.035040 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:58.035082 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:58.038630 (+ 3548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:58.039428 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:58.039437 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:58.039441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:58.039562 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:58.039941 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:58.039946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:58.039950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:58.040292 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:58.040578 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:58.040889 (+ 311us) server_negotiation.cc:299] Negotiation successful
0925 11:00:58.041151 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":226,"threads_started":1}
I20260925 11:00:58.042841 29156 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:58.126933 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 29033
I20260925 11:00:58.139322 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 29166
Sep 25 11:00:58 dist-test-slave-trms krb5kdc[28793](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:00:58.154703 29308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:58.140181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.24.159.195:34957)
0925 11:00:58.140760 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:58.140766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:58.140796 (+ 30us) server_negotiation.cc:407] Connection header received
0925 11:00:58.140855 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:58.140861 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:58.141076 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:58.141210 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:58.142355 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:58.143363 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:58.144591 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:58.145141 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:58.145231 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:58.149514 (+ 4283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:58.149527 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:58.149543 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:58.149582 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:58.152284 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:58.152917 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:58.152924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:58.152926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:58.153047 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:58.153351 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:58.153354 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:58.153356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:58.153695 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:58.153933 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:58.154142 (+ 209us) server_negotiation.cc:299] Negotiation successful
0925 11:00:58.154430 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":279,"threads_started":1}
I20260925 11:00:58.155963 29289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:58.236536 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 29166
I20260925 11:00:58.249419 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 28806
I20260925 11:00:58.250538 28868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:00:58.352479 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 28806
2026-09-25T11:00:58Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3660 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:00:58 dist-test-slave-trms krb5kdc[29313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:00:58 dist-test-slave-trms krb5kdc[29313](info): set up 2 sockets
Sep 25 11:00:58 dist-test-slave-trms krb5kdc[29313](info): commencing operation
krb5kdc: starting...
W20260925 11:00:59.334617 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.946s user 0.007s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:00:59 dist-test-slave-trms krb5kdc[29313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:00:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:00:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:00:59.515558 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44611
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:44611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:00:59.823967 29326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:00:59.824245 29326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:00:59.824334 29326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:00:59.831800 29326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:00:59.831915 29326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:00:59.831964 29326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:00:59.832019 29326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:00:59.832065 29326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:00:59.841858 29326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:44611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:00:59.843144 29326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:00:59.844359 29326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:00:59.855139 29331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:59.855062 29334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:00:59.855095 29332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:00:59.855823 29326 server_base.cc:1034] running on GCE node
I20260925 11:00:59.856572 29326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:00:59.857889 29326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:00:59.859076 29326 hybrid_clock.cc:648] HybridClock initialized: now 1790334059859030 us; error 37 us; skew 500 ppm
I20260925 11:00:59.859395 29326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:00:59 dist-test-slave-trms krb5kdc[29313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:00:59.862313 29326 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:00:59.863422 29326 fs_manager.cc:359] Metadata directory not provided
I20260925 11:00:59.863509 29326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:00:59.863772 29326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:00:59.866648 29326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "f48ab3d4cd88409da44ce01ac371516f"
format_stamp: "Formatted at 2026-09-25 11:00:59 on dist-test-slave-trms"
server_key: "bbeef3046baf13ef3793cba02182f297"
server_key_iv: "b542cea592834685f69db61abe0552a2"
server_key_version: "encryptionkey@0"
I20260925 11:00:59.867370 29326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "f48ab3d4cd88409da44ce01ac371516f"
format_stamp: "Formatted at 2026-09-25 11:00:59 on dist-test-slave-trms"
server_key: "bbeef3046baf13ef3793cba02182f297"
server_key_iv: "b542cea592834685f69db61abe0552a2"
server_key_version: "encryptionkey@0"
I20260925 11:00:59.872087 29326 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 11:00:59.875356 29340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:00:59.876593 29326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:00:59.876744 29326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "f48ab3d4cd88409da44ce01ac371516f"
format_stamp: "Formatted at 2026-09-25 11:00:59 on dist-test-slave-trms"
server_key: "bbeef3046baf13ef3793cba02182f297"
server_key_iv: "b542cea592834685f69db61abe0552a2"
server_key_version: "encryptionkey@0"
I20260925 11:00:59.876940 29326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:00:59.895200 29326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:00:59.899271 29326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:00:59.899508 29326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:00:59.916041 29326 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:44611
I20260925 11:00:59.916059 29391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:44611 every 8 connection(s)
I20260925 11:00:59.916925 29326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:00:59.920437 29392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:00:59.921089 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 29326
I20260925 11:00:59.921267 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:00:59.921855 25215 external_mini_cluster.cc:1428] Setting key 91c4d92e418539c51db9e18a0ba8d8bd
I20260925 11:00:59.933548 29392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f: Bootstrap starting.
Sep 25 11:00:59 dist-test-slave-trms krb5kdc[29313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:00:59.937284 29392 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f: Neither blocks nor log segments found. Creating new log.
I20260925 11:00:59.938452 29392 log.cc:824] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f: Log is configured to *not* fsync() on all Append() calls
I20260925 11:00:59.941571 29392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f: No bootstrap required, opened a new log
I20260925 11:00:59.944023 29395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:00:59.923727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.24.159.254:44611)
0925 11:00:59.924463 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0925 11:00:59.924475 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:00:59.924523 (+ 48us) server_negotiation.cc:407] Connection header received
0925 11:00:59.926098 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:00:59.926120 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:00:59.926563 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:00:59.926882 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:00:59.928524 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:59.929890 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:59.931621 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:00:59.932272 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:00:59.932409 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:00:59.936872 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:00:59.936880 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:00:59.936895 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:00:59.936933 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:00:59.940103 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:59.940850 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:59.940855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:59.940861 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:59.941013 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:00:59.941405 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:00:59.941408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:00:59.941410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:00:59.941967 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:00:59.942218 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0925 11:00:59.942625 (+ 407us) server_negotiation.cc:299] Negotiation successful
0925 11:00:59.942961 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":339,"threads_started":1}
I20260925 11:00:59.949126 29392 raft_consensus.cc:348] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ab3d4cd88409da44ce01ac371516f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44611 } }
I20260925 11:00:59.949465 29392 raft_consensus.cc:374] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:00:59.949555 29392 raft_consensus.cc:729] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f48ab3d4cd88409da44ce01ac371516f, State: Initialized, Role: FOLLOWER
I20260925 11:00:59.950063 29392 consensus_queue.cc:260] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ab3d4cd88409da44ce01ac371516f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44611 } }
I20260925 11:00:59.950290 29392 raft_consensus.cc:388] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:00:59.950420 29392 raft_consensus.cc:482] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:00:59.950594 29392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:00:59.952670 29392 raft_consensus.cc:504] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ab3d4cd88409da44ce01ac371516f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44611 } }
I20260925 11:00:59.953119 29392 leader_election.cc:302] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f48ab3d4cd88409da44ce01ac371516f; no voters:
I20260925 11:00:59.953694 29392 leader_election.cc:288] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:00:59.953831 29397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:00:59.954545 29397 raft_consensus.cc:686] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [term 1 LEADER]: Becoming Leader. State: Replica: f48ab3d4cd88409da44ce01ac371516f, State: Running, Role: LEADER
I20260925 11:00:59.955080 29397 consensus_queue.cc:237] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ab3d4cd88409da44ce01ac371516f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44611 } }
I20260925 11:00:59.955772 29392 sys_catalog.cc:564] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:00:59.957799 29399 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f48ab3d4cd88409da44ce01ac371516f. Latest consensus state: current_term: 1 leader_uuid: "f48ab3d4cd88409da44ce01ac371516f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ab3d4cd88409da44ce01ac371516f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44611 } } }
I20260925 11:00:59.957777 29398 sys_catalog.cc:455] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f48ab3d4cd88409da44ce01ac371516f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f48ab3d4cd88409da44ce01ac371516f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44611 } } }
I20260925 11:00:59.958299 29398 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [sys.catalog]: This master's current role is: LEADER
I20260925 11:00:59.960781 29399 sys_catalog.cc:458] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f [sys.catalog]: This master's current role is: LEADER
W20260925 11:00:59.965646 29413 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:00:59.965737 29413 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:00:59.965880 29406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:00:59.967479 29406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:00:59.974923 29406 catalog_manager.cc:1347] Generated new cluster ID: 23f1b3e8bf594bac908e74d4b0ceb742
I20260925 11:00:59.975028 29406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:00:59.998742 29406 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:00.000036 29406 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:00.005546 29406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f: Generated new TSK 0
I20260925 11:01:00.006362 29406 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:01:00.053107 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44611
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:00.360584 29418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:00.360877 29418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:00.361013 29418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:00.368043 29418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:00.368166 29418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:00.368322 29418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:00.377831 29418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:00.379138 29418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:00.380441 29418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:00.392465 29423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:00.392395 29426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:00.392433 29424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:00.393224 29418 server_base.cc:1034] running on GCE node
I20260925 11:01:00.393793 29418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:00.394735 29418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:00.395927 29418 hybrid_clock.cc:648] HybridClock initialized: now 1790334060395875 us; error 47 us; skew 500 ppm
I20260925 11:01:00.396221 29418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:00 dist-test-slave-trms krb5kdc[29313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334060, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:00.399137 29418 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:01:00.400238 29418 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:00.400336 29418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:00.400626 29418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:00.403604 29418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "43efaa7361d54b46b4c93d37cb99a3ad"
format_stamp: "Formatted at 2026-09-25 11:01:00 on dist-test-slave-trms"
server_key: "66a097e0edf86fda65fb638c5368c637"
server_key_iv: "54496996e5359eca010baa1b1c508195"
server_key_version: "encryptionkey@0"
I20260925 11:01:00.404431 29418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "43efaa7361d54b46b4c93d37cb99a3ad"
format_stamp: "Formatted at 2026-09-25 11:01:00 on dist-test-slave-trms"
server_key: "66a097e0edf86fda65fb638c5368c637"
server_key_iv: "54496996e5359eca010baa1b1c508195"
server_key_version: "encryptionkey@0"
I20260925 11:01:00.409168 29418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 11:01:00.412514 29432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:00.413797 29418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 11:01:00.413954 29418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "43efaa7361d54b46b4c93d37cb99a3ad"
format_stamp: "Formatted at 2026-09-25 11:01:00 on dist-test-slave-trms"
server_key: "66a097e0edf86fda65fb638c5368c637"
server_key_iv: "54496996e5359eca010baa1b1c508195"
server_key_version: "encryptionkey@0"
I20260925 11:01:00.414148 29418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:00.426352 29418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:00.430500 29418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:00.430747 29418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:00.431674 29418 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:00.432945 29418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:00.433040 29418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:00.433143 29418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:00.433214 29418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:00.461047 29418 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:35147
I20260925 11:01:00.461077 29544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:35147 every 8 connection(s)
I20260925 11:01:00.461930 29418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:00.471257 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 29418
I20260925 11:01:00.471443 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:01:00.472026 25215 external_mini_cluster.cc:1428] Setting key 4c8abdcac7d245f04fd149a67942ec1d
Sep 25 11:01:00 dist-test-slave-trms krb5kdc[29313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334060, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:00.480667 29547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:00.464216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:35727 (local address 127.24.159.254:44611)
0925 11:01:00.464628 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:00.464632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:00.465492 (+ 860us) server_negotiation.cc:407] Connection header received
0925 11:01:00.467299 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:00.467305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:00.467390 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:00.467481 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:00.468977 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:00.469697 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:00.471263 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:00.471705 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:00.471816 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:00.475317 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:00.475321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:00.475325 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:00.475357 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:00.477637 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:00.478349 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:00.478354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:00.478358 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:00.478522 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:00.478936 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:00.478940 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:00.478942 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:00.479213 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:00.479341 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:00.480062 (+ 721us) server_negotiation.cc:299] Negotiation successful
0925 11:01:00.480284 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":222,"threads_started":1}
I20260925 11:01:00.481585 29548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:00.464688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44611 (local address 127.24.159.193:35727)
0925 11:01:00.465339 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:00.465381 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:00.467042 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:00.467721 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:00.467730 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:00.468232 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:00.468809 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:00.468827 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:00.469834 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:00.469837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:00.471131 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:00.471144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:00.471930 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:00.471938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:00.472162 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:00.472803 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:00.472829 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:00.475117 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:00.477777 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:00.477785 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:00.477801 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:00.478217 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:00.478642 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:00.478647 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:00.478650 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:00.478820 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:00.479336 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:00.479344 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:00.479708 (+ 364us) client_negotiation.cc:769] Sending connection context
0925 11:01:00.479964 (+ 256us) client_negotiation.cc:240] Negotiation successful
0925 11:01:00.480478 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":251,"threads_started":1}
I20260925 11:01:00.483235 29545 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44611
I20260925 11:01:00.483561 29545 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:00.484287 29545 heartbeater.cc:511] Master 127.24.159.254:44611 requested a full tablet report, sending...
I20260925 11:01:00.486505 29357 ts_manager.cc:194] Registered new tserver with Master: 43efaa7361d54b46b4c93d37cb99a3ad (127.24.159.193:35147)
I20260925 11:01:00.487990 29357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:35727
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:01:00.511503 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44611
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:00.819077 29551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:00.819353 29551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:00.819494 29551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:00.826586 29551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:00.826702 29551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:00.826861 29551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:00.836476 29551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:00.837775 29551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:00.839089 29551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:00.851218 29556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:00.851250 29557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:00.851177 29559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:00.852454 29551 server_base.cc:1034] running on GCE node
I20260925 11:01:00.853031 29551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:00.853945 29551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:00.855142 29551 hybrid_clock.cc:648] HybridClock initialized: now 1790334060855086 us; error 69 us; skew 500 ppm
I20260925 11:01:00.855477 29551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:00 dist-test-slave-trms krb5kdc[29313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334060, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:00.858440 29551 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:01:00.859611 29551 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:00.859714 29551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:00.860005 29551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:00.862960 29551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "413239b540394ba1aa3aff717e6f4ca6"
format_stamp: "Formatted at 2026-09-25 11:01:00 on dist-test-slave-trms"
server_key: "824c434ba479773bc8adc7917f081877"
server_key_iv: "819a4d7abf91b9628e56d21b1c07cd5f"
server_key_version: "encryptionkey@0"
I20260925 11:01:00.863724 29551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "413239b540394ba1aa3aff717e6f4ca6"
format_stamp: "Formatted at 2026-09-25 11:01:00 on dist-test-slave-trms"
server_key: "824c434ba479773bc8adc7917f081877"
server_key_iv: "819a4d7abf91b9628e56d21b1c07cd5f"
server_key_version: "encryptionkey@0"
I20260925 11:01:00.868716 29551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:01:00.872110 29565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:00.873383 29551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:01:00.873538 29551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "413239b540394ba1aa3aff717e6f4ca6"
format_stamp: "Formatted at 2026-09-25 11:01:00 on dist-test-slave-trms"
server_key: "824c434ba479773bc8adc7917f081877"
server_key_iv: "819a4d7abf91b9628e56d21b1c07cd5f"
server_key_version: "encryptionkey@0"
I20260925 11:01:00.873734 29551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:00.897183 29551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:00.901007 29551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:00.901257 29551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:00.902220 29551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:00.903549 29551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:00.903646 29551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:00.903744 29551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:00.903810 29551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:00.934123 29551 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:42839
I20260925 11:01:00.934211 29677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:42839 every 8 connection(s)
I20260925 11:01:00.934996 29551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:00.938655 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 29551
I20260925 11:01:00.938843 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:01:00.939340 25215 external_mini_cluster.cc:1428] Setting key a86669618e535d11e287edbb5522325d
Sep 25 11:01:00 dist-test-slave-trms krb5kdc[29313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334060, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:00.955039 29547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:00.937252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:54369 (local address 127.24.159.254:44611)
0925 11:01:00.937468 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:00.937475 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:00.938709 (+ 1234us) server_negotiation.cc:407] Connection header received
0925 11:01:00.940956 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:00.940962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:00.941043 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:00.941124 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:00.942680 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:00.943362 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:00.944894 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:00.945296 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:00.945402 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:00.949383 (+ 3981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:00.949389 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:00.949392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:00.949424 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:00.951764 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:00.952632 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:00.952636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:00.952638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:00.952761 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:00.953241 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:00.953244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:00.953246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:00.953511 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:00.953616 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:00.954607 (+ 991us) server_negotiation.cc:299] Negotiation successful
0925 11:01:00.954792 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 11:01:00.956699 29680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:00.937737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44611 (local address 127.24.159.194:54369)
0925 11:01:00.938523 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:00.938573 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:00.940693 (+ 2120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:00.941364 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:00.941372 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:00.941876 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:00.942520 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:00.942538 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:00.943508 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:00.943512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:00.944757 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:00.944769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:00.945505 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:00.945512 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:00.945797 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:00.946535 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:00.946564 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:00.949155 (+ 2591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:00.951921 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:00.951939 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:00.951959 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:00.952526 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:00.952876 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:00.952881 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:00.952885 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:00.953136 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:00.953641 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:00.953650 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:00.954159 (+ 509us) client_negotiation.cc:769] Sending connection context
0925 11:01:00.954526 (+ 367us) client_negotiation.cc:240] Negotiation successful
0925 11:01:00.955117 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":288,"threads_started":1}
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
I20260925 11:01:00.958235 29678 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44611
I20260925 11:01:00.958561 29678 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:00.959255 29678 heartbeater.cc:511] Master 127.24.159.254:44611 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
I20260925 11:01:00.960683 29357 ts_manager.cc:194] Registered new tserver with Master: 413239b540394ba1aa3aff717e6f4ca6 (127.24.159.194:42839)
I20260925 11:01:00.961467 29357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:54369
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:01:00.980445 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44611
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37885
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:01.291538 29683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:01.291839 29683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:01.291985 29683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:01.299062 29683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:01.299178 29683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:01.299330 29683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:01.308663 29683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37885
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44611
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:01.309921 29683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:01.311198 29683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:01.323453 29688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:01.323398 29689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:01.323376 29691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:01.324347 29683 server_base.cc:1034] running on GCE node
I20260925 11:01:01.324944 29683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:01.325836 29683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:01.327028 29683 hybrid_clock.cc:648] HybridClock initialized: now 1790334061326981 us; error 36 us; skew 500 ppm
I20260925 11:01:01.327319 29683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:01 dist-test-slave-trms krb5kdc[29313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334061, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:01.333335 29683 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:01:01.334537 29683 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:01.334630 29683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:01.334923 29683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:01.337750 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "0936bd7a7b9544cf911ae17b05d068b7"
format_stamp: "Formatted at 2026-09-25 11:01:01 on dist-test-slave-trms"
server_key: "73a972cb9451f821d68ccf1e14ecf56d"
server_key_iv: "a1bc5525ac09b6304b09d1ffb2ef3d3c"
server_key_version: "encryptionkey@0"
I20260925 11:01:01.338621 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "0936bd7a7b9544cf911ae17b05d068b7"
format_stamp: "Formatted at 2026-09-25 11:01:01 on dist-test-slave-trms"
server_key: "73a972cb9451f821d68ccf1e14ecf56d"
server_key_iv: "a1bc5525ac09b6304b09d1ffb2ef3d3c"
server_key_version: "encryptionkey@0"
I20260925 11:01:01.343405 29683 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260925 11:01:01.346640 29697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:01.347926 29683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:01:01.348073 29683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "0936bd7a7b9544cf911ae17b05d068b7"
format_stamp: "Formatted at 2026-09-25 11:01:01 on dist-test-slave-trms"
server_key: "73a972cb9451f821d68ccf1e14ecf56d"
server_key_iv: "a1bc5525ac09b6304b09d1ffb2ef3d3c"
server_key_version: "encryptionkey@0"
I20260925 11:01:01.348268 29683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:01.378420 29683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:01.382484 29683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:01.382745 29683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:01.383674 29683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:01.384964 29683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:01.385059 29683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:01.385174 29683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:01.385247 29683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:01.414155 29683 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:38833
I20260925 11:01:01.414223 29809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:38833 every 8 connection(s)
I20260925 11:01:01.414991 29683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:01.415946 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 29683
I20260925 11:01:01.416141 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:01:01.416656 25215 external_mini_cluster.cc:1428] Setting key 598358e1be7bd20bfca6e5343ec6df47
Sep 25 11:01:01 dist-test-slave-trms krb5kdc[29313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334061, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:01.433971 29547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.417697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:60527 (local address 127.24.159.254:44611)
0925 11:01:01.417865 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.417870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.419204 (+ 1334us) server_negotiation.cc:407] Connection header received
0925 11:01:01.420997 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.421001 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.421066 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.421142 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:01.422607 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.423323 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.424765 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.425165 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.425248 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.428384 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:01.428389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:01.428392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:01.428423 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:01.430966 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:01.431686 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:01.431690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:01.431693 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:01.431808 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:01.432235 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:01.432240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:01.432243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:01.432536 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:01.432665 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.433387 (+ 722us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.433618 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 11:01:01.435026 29812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.418217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44611 (local address 127.24.159.195:60527)
0925 11:01:01.419039 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:01.419096 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:01.420748 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:01.421340 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:01.421348 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:01.421854 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:01.422467 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.422486 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.423421 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.423431 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.424660 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.424672 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.425329 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.425334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.425526 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.426095 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:01.426116 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:01.428175 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:01.431126 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:01.431133 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:01.431147 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:01.431561 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:01.431909 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:01.431914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:01.431916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:01.432131 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:01.432658 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:01.432667 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:01.433019 (+ 352us) client_negotiation.cc:769] Sending connection context
0925 11:01:01.433271 (+ 252us) client_negotiation.cc:240] Negotiation successful
0925 11:01:01.433785 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":342,"threads_started":1}
I20260925 11:01:01.436345 29810 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44611
I20260925 11:01:01.436626 29810 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:01.437227 29810 heartbeater.cc:511] Master 127.24.159.254:44611 requested a full tablet report, sending...
I20260925 11:01:01.438580 29357 ts_manager.cc:194] Registered new tserver with Master: 0936bd7a7b9544cf911ae17b05d068b7 (127.24.159.195:38833)
I20260925 11:01:01.439317 29357 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:60527
I20260925 11:01:01.443718 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:01:01.459268 29547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.447922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.24.159.254:44611)
0925 11:01:01.448071 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.448075 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.448579 (+ 504us) server_negotiation.cc:407] Connection header received
0925 11:01:01.448723 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.448726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.448783 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.448858 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:01.450083 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.450802 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.452022 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.452473 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.452572 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.454644 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:01.454648 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:01.454651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:01.454681 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:01.457051 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:01.457584 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:01.457588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:01.457590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:01.457714 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:01.457974 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:01.457977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:01.457979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:01.458336 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:01.458538 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.458805 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.458960 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 11:01:01.463320 29357 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:01:01.469983 29357 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:01.490942 29545 heartbeater.cc:503] Master 127.24.159.254:44611 was elected leader, sending a full tablet report...
I20260925 11:01:01.492450 29823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.481419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:42839 (local address 127.0.0.1:38948)
0925 11:01:01.482462 (+ 1043us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:01.482512 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:01.482694 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:01.484414 (+ 1720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:01.484422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:01.484455 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:01.484918 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.484943 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.486820 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.486825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.489593 (+ 2768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.489604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.491382 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.491389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.491583 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.491643 (+ 60us) client_negotiation.cc:769] Sending connection context
0925 11:01:01.491773 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 11:01:01.491973 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":716,"thread_start_us":1763,"threads_started":1}
I20260925 11:01:01.492918 29825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.481950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58130 (local address 127.24.159.193:35147)
0925 11:01:01.483201 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.483211 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.483544 (+ 333us) server_negotiation.cc:407] Connection header received
0925 11:01:01.483803 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.483809 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.483989 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.484148 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:01.485094 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.486668 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.489561 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.491484 (+ 1923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.491670 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.491825 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.492430 (+ 605us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.492592 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1090,"thread_start_us":965,"threads_started":1}
I20260925 11:01:01.493134 29824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.481907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38948 (local address 127.24.159.194:42839)
0925 11:01:01.483577 (+ 1670us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.483585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.483606 (+ 21us) server_negotiation.cc:407] Connection header received
0925 11:01:01.483702 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.483710 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.483985 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.484156 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:01.485099 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.486668 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.489732 (+ 3064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.491171 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.491343 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.491537 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.492677 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.492853 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1533,"thread_start_us":1697,"threads_started":1}
I20260925 11:01:01.493731 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.481742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:35147 (local address 127.0.0.1:58130)
0925 11:01:01.483439 (+ 1697us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:01.483456 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:01.483646 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:01.484414 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:01.484422 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:01.484471 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:01.484884 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.484897 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.486827 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.486831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.489375 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.489391 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.491995 (+ 2604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.492002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.492204 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.492237 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 11:01:01.493095 (+ 858us) client_negotiation.cc:240] Negotiation successful
0925 11:01:01.493338 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1571,"mutex_wait_us":46,"spinlock_wait_cycles":4992,"thread_start_us":335,"threads_started":1}
I20260925 11:01:01.496958 29613 tablet_service.cc:1467] Processing CreateTablet for tablet 0249544f3a21468b921e816c60d1c0a9 (DEFAULT_TABLE table=test-table [id=f6e5f878798843bfbc3ac0be12181134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:01.497535 29480 tablet_service.cc:1467] Processing CreateTablet for tablet 0249544f3a21468b921e816c60d1c0a9 (DEFAULT_TABLE table=test-table [id=f6e5f878798843bfbc3ac0be12181134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:01.498492 29613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0249544f3a21468b921e816c60d1c0a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:01.498778 29480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0249544f3a21468b921e816c60d1c0a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:01.499023 29826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.483482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:38833 (local address 127.0.0.1:37190)
0925 11:01:01.487327 (+ 3845us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:01.487344 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:01.487448 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:01.491874 (+ 4426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:01.491878 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:01.491902 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:01.492288 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.492299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.494592 (+ 2293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.494597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.496573 (+ 1976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.496589 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.498300 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.498307 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.498479 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.498510 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 11:01:01.498589 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 11:01:01.498715 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3699,"thread_start_us":3542,"threads_started":1}
I20260925 11:01:01.499317 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.483765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37190 (local address 127.24.159.195:38833)
0925 11:01:01.490164 (+ 6399us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.490173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.490239 (+ 66us) server_negotiation.cc:407] Connection header received
0925 11:01:01.490333 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.490341 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.490608 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.492737 (+ 2129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:01.493018 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.494464 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.496747 (+ 2283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.498088 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.498289 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.498805 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.498873 (+ 68us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.499023 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6222,"thread_start_us":6068,"threads_started":1}
I20260925 11:01:01.506345 29745 tablet_service.cc:1467] Processing CreateTablet for tablet 0249544f3a21468b921e816c60d1c0a9 (DEFAULT_TABLE table=test-table [id=f6e5f878798843bfbc3ac0be12181134]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:01.507797 29745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0249544f3a21468b921e816c60d1c0a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:01.510241 29828 tablet_bootstrap.cc:492] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: Bootstrap starting.
I20260925 11:01:01.514086 29828 tablet_bootstrap.cc:654] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:01.515488 29828 log.cc:824] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:01.516927 29829 tablet_bootstrap.cc:492] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: Bootstrap starting.
I20260925 11:01:01.518672 29828 tablet_bootstrap.cc:492] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: No bootstrap required, opened a new log
I20260925 11:01:01.518945 29828 ts_tablet_manager.cc:1397] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260925 11:01:01.521142 29829 tablet_bootstrap.cc:654] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:01.522471 29829 log.cc:824] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:01.523530 29831 tablet_bootstrap.cc:492] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: Bootstrap starting.
I20260925 11:01:01.526770 29829 tablet_bootstrap.cc:492] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: No bootstrap required, opened a new log
I20260925 11:01:01.527036 29829 ts_tablet_manager.cc:1397] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260925 11:01:01.527072 29831 tablet_bootstrap.cc:654] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:01.527792 29828 raft_consensus.cc:348] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } }
I20260925 11:01:01.528259 29828 raft_consensus.cc:374] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:01.528276 29831 log.cc:824] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:01.528414 29828 raft_consensus.cc:729] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413239b540394ba1aa3aff717e6f4ca6, State: Initialized, Role: FOLLOWER
I20260925 11:01:01.529117 29828 consensus_queue.cc:260] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } }
I20260925 11:01:01.530313 29678 heartbeater.cc:503] Master 127.24.159.254:44611 was elected leader, sending a full tablet report...
I20260925 11:01:01.531028 29828 ts_tablet_manager.cc:1428] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: Time spent starting tablet: real 0.012s user 0.005s sys 0.007s
I20260925 11:01:01.535290 29829 raft_consensus.cc:348] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } }
I20260925 11:01:01.535622 29829 raft_consensus.cc:374] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:01.535714 29829 raft_consensus.cc:729] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43efaa7361d54b46b4c93d37cb99a3ad, State: Initialized, Role: FOLLOWER
I20260925 11:01:01.536286 29829 consensus_queue.cc:260] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } }
I20260925 11:01:01.537043 29831 tablet_bootstrap.cc:492] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: No bootstrap required, opened a new log
I20260925 11:01:01.537349 29831 ts_tablet_manager.cc:1397] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
I20260925 11:01:01.537936 29829 ts_tablet_manager.cc:1428] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: Time spent starting tablet: real 0.011s user 0.006s sys 0.005s
I20260925 11:01:01.545342 29831 raft_consensus.cc:348] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } }
I20260925 11:01:01.545689 29831 raft_consensus.cc:374] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:01.545779 29831 raft_consensus.cc:729] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0936bd7a7b9544cf911ae17b05d068b7, State: Initialized, Role: FOLLOWER
I20260925 11:01:01.546378 29831 consensus_queue.cc:260] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } }
I20260925 11:01:01.547312 29810 heartbeater.cc:503] Master 127.24.159.254:44611 was elected leader, sending a full tablet report...
I20260925 11:01:01.547986 29831 ts_tablet_manager.cc:1428] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260925 11:01:01.616571 29834 raft_consensus.cc:482] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:01.616871 29834 raft_consensus.cc:504] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } }
I20260925 11:01:01.618657 29834 leader_election.cc:288] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43efaa7361d54b46b4c93d37cb99a3ad (127.24.159.193:35147), 0936bd7a7b9544cf911ae17b05d068b7 (127.24.159.195:38833)
I20260925 11:01:01.627162 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.619078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:59983 (local address 127.24.159.195:38833)
0925 11:01:01.619248 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.619253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.619936 (+ 683us) server_negotiation.cc:407] Connection header received
0925 11:01:01.620147 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.620152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.620216 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.620295 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:01.620947 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.622024 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.624817 (+ 2793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.626004 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.626139 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.626238 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.626786 (+ 548us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.626917 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:01:01.627167 29825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.619038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:45133 (local address 127.24.159.193:35147)
0925 11:01:01.619239 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.619245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.619952 (+ 707us) server_negotiation.cc:407] Connection header received
0925 11:01:01.620115 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.620120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.620197 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.620296 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:01.621055 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.622046 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.624814 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.625976 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.626139 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.626238 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.626786 (+ 548us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.626919 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:01:01.627362 29838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.619117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:35147 (local address 127.24.159.194:45133)
0925 11:01:01.619863 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:01.619875 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:01.619989 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:01.620315 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:01.620320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:01.620359 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:01.620913 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.620934 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.622148 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.622152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.624669 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.624678 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.626261 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.626267 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.626496 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.626656 (+ 160us) client_negotiation.cc:769] Sending connection context
0925 11:01:01.626751 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 11:01:01.626937 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":331,"spinlock_wait_cycles":3968,"thread_start_us":386,"threads_started":1}
I20260925 11:01:01.627602 29837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.618984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:38833 (local address 127.24.159.194:59983)
0925 11:01:01.619813 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:01.619833 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:01.620032 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:01.620315 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:01.620320 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:01.620350 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:01.620810 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.620822 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.622152 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.622157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.624660 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:01.624677 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.626257 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.626264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:01.626495 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.626656 (+ 161us) client_negotiation.cc:769] Sending connection context
0925 11:01:01.626751 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 11:01:01.626927 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":252,"spinlock_wait_cycles":42240,"thread_start_us":434,"threads_started":1}
I20260925 11:01:01.628513 29500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0249544f3a21468b921e816c60d1c0a9" candidate_uuid: "413239b540394ba1aa3aff717e6f4ca6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" is_pre_election: true
I20260925 11:01:01.628785 29765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0249544f3a21468b921e816c60d1c0a9" candidate_uuid: "413239b540394ba1aa3aff717e6f4ca6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0936bd7a7b9544cf911ae17b05d068b7" is_pre_election: true
I20260925 11:01:01.629007 29500 raft_consensus.cc:2454] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413239b540394ba1aa3aff717e6f4ca6 in term 0.
I20260925 11:01:01.629240 29765 raft_consensus.cc:2454] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413239b540394ba1aa3aff717e6f4ca6 in term 0.
I20260925 11:01:01.629739 29568 leader_election.cc:302] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413239b540394ba1aa3aff717e6f4ca6, 43efaa7361d54b46b4c93d37cb99a3ad; no voters:
I20260925 11:01:01.630136 29834 raft_consensus.cc:2781] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:01.630294 29834 raft_consensus.cc:482] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:01.630369 29834 raft_consensus.cc:3037] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:01.632519 29834 raft_consensus.cc:504] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } }
I20260925 11:01:01.633092 29834 leader_election.cc:288] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [CANDIDATE]: Term 1 election: Requested vote from peers 43efaa7361d54b46b4c93d37cb99a3ad (127.24.159.193:35147), 0936bd7a7b9544cf911ae17b05d068b7 (127.24.159.195:38833)
I20260925 11:01:01.633478 29500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0249544f3a21468b921e816c60d1c0a9" candidate_uuid: "413239b540394ba1aa3aff717e6f4ca6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43efaa7361d54b46b4c93d37cb99a3ad"
I20260925 11:01:01.633503 29765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0249544f3a21468b921e816c60d1c0a9" candidate_uuid: "413239b540394ba1aa3aff717e6f4ca6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0936bd7a7b9544cf911ae17b05d068b7"
I20260925 11:01:01.633656 29500 raft_consensus.cc:3037] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:01.633654 29765 raft_consensus.cc:3037] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:01.635809 29765 raft_consensus.cc:2454] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413239b540394ba1aa3aff717e6f4ca6 in term 1.
I20260925 11:01:01.635818 29500 raft_consensus.cc:2454] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413239b540394ba1aa3aff717e6f4ca6 in term 1.
I20260925 11:01:01.636175 29566 leader_election.cc:302] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0936bd7a7b9544cf911ae17b05d068b7, 413239b540394ba1aa3aff717e6f4ca6; no voters:
I20260925 11:01:01.636469 29834 raft_consensus.cc:2781] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:01.636871 29834 raft_consensus.cc:686] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 1 LEADER]: Becoming Leader. State: Replica: 413239b540394ba1aa3aff717e6f4ca6, State: Running, Role: LEADER
I20260925 11:01:01.637326 29834 consensus_queue.cc:237] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } }
I20260925 11:01:01.641968 29356 catalog_manager.cc:5495] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 413239b540394ba1aa3aff717e6f4ca6 (127.24.159.194). New cstate: current_term: 1 leader_uuid: "413239b540394ba1aa3aff717e6f4ca6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "413239b540394ba1aa3aff717e6f4ca6" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 } health_report { overall_health: UNKNOWN } } }
W20260925 11:01:01.666472 29811 tablet.cc:2369] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:01:01.686718 29679 tablet.cc:2369] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:01:01.693876 29824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.688667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38960 (local address 127.24.159.194:42839)
0925 11:01:01.688891 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.688897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.688984 (+ 87us) server_negotiation.cc:407] Connection header received
0925 11:01:01.689199 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.689203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.689279 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.689406 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:01.689776 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.690524 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.691822 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.692229 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.692337 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.692802 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:01.693309 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:01.693446 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.693546 (+ 100us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.693622 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 11:01:01.701555 29500 raft_consensus.cc:1261] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [term 1 FOLLOWER]: Refusing update from remote peer 413239b540394ba1aa3aff717e6f4ca6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:01.701551 29765 raft_consensus.cc:1261] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [term 1 FOLLOWER]: Refusing update from remote peer 413239b540394ba1aa3aff717e6f4ca6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:01.702476 29839 consensus_queue.cc:1035] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "43efaa7361d54b46b4c93d37cb99a3ad" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:01.702793 29834 consensus_queue.cc:1035] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0936bd7a7b9544cf911ae17b05d068b7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:01.713119 29842 mvcc.cc:205] Tried to move back new op lower bound from 7333208316720660480 to 7333208316470239232. Current Snapshot: MvccSnapshot[applied={T|T < 7333208316720660480}]
W20260925 11:01:01.714275 29546 tablet.cc:2369] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:01:01.727806 29355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:01.728137 29355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:01.731880 29355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f48ab3d4cd88409da44ce01ac371516f: Sending DeleteTablet for 3 replicas of tablet 0249544f3a21468b921e816c60d1c0a9
I20260925 11:01:01.733040 29480 tablet_service.cc:1514] Processing DeleteTablet for tablet 0249544f3a21468b921e816c60d1c0a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:01 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:58130
I20260925 11:01:01.733088 29613 tablet_service.cc:1514] Processing DeleteTablet for tablet 0249544f3a21468b921e816c60d1c0a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:01 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38948
I20260925 11:01:01.733457 29745 tablet_service.cc:1514] Processing DeleteTablet for tablet 0249544f3a21468b921e816c60d1c0a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:01 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:37190
I20260925 11:01:01.737974 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 29418
I20260925 11:01:01.745178 29854 tablet_replica.cc:330] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: stopping tablet replica
I20260925 11:01:01.745780 29854 raft_consensus.cc:2229] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:01.746452 29854 raft_consensus.cc:2258] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:01.748792 29854 ts_tablet_manager.cc:1905] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:01.756040 29854 ts_tablet_manager.cc:1918] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:01.756336 29854 log.cc:1196] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/0249544f3a21468b921e816c60d1c0a9
I20260925 11:01:01.756951 29854 ts_tablet_manager.cc:1939] T 0249544f3a21468b921e816c60d1c0a9 P 0936bd7a7b9544cf911ae17b05d068b7: Deleting consensus metadata
I20260925 11:01:01.758988 29341 catalog_manager.cc:4841] TS 0936bd7a7b9544cf911ae17b05d068b7 (127.24.159.195:38833): tablet 0249544f3a21468b921e816c60d1c0a9 (table test-table [id=f6e5f878798843bfbc3ac0be12181134]) successfully deleted
I20260925 11:01:01.760138 29852 tablet_replica.cc:330] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: stopping tablet replica
I20260925 11:01:01.760182 29853 tablet_replica.cc:330] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: stopping tablet replica
I20260925 11:01:01.760653 29852 raft_consensus.cc:2229] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:01.760643 29853 raft_consensus.cc:2229] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:01:01.761228 29852 raft_consensus.cc:2258] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:01.761355 29853 raft_consensus.cc:2258] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:01.763528 29852 ts_tablet_manager.cc:1905] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:01.763536 29853 ts_tablet_manager.cc:1905] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 11:01:01 dist-test-slave-trms krb5kdc[29313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:01:01.770563 29852 ts_tablet_manager.cc:1918] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:01.770862 29852 log.cc:1196] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/0249544f3a21468b921e816c60d1c0a9
I20260925 11:01:01.771456 29852 ts_tablet_manager.cc:1939] T 0249544f3a21468b921e816c60d1c0a9 P 43efaa7361d54b46b4c93d37cb99a3ad: Deleting consensus metadata
I20260925 11:01:01.771438 29853 ts_tablet_manager.cc:1918] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:01.771602 29853 log.cc:1196] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/0249544f3a21468b921e816c60d1c0a9
I20260925 11:01:01.772406 29853 ts_tablet_manager.cc:1939] T 0249544f3a21468b921e816c60d1c0a9 P 413239b540394ba1aa3aff717e6f4ca6: Deleting consensus metadata
I20260925 11:01:01.773593 29343 catalog_manager.cc:4841] TS 43efaa7361d54b46b4c93d37cb99a3ad (127.24.159.193:35147): tablet 0249544f3a21468b921e816c60d1c0a9 (table test-table [id=f6e5f878798843bfbc3ac0be12181134]) successfully deleted
I20260925 11:01:01.774089 29341 catalog_manager.cc:4841] TS 413239b540394ba1aa3aff717e6f4ca6 (127.24.159.194:42839): tablet 0249544f3a21468b921e816c60d1c0a9 (table test-table [id=f6e5f878798843bfbc3ac0be12181134]) successfully deleted
I20260925 11:01:01.776787 29825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.739152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58144 (local address 127.24.159.193:35147)
0925 11:01:01.739337 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.739344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.759223 (+ 19879us) server_negotiation.cc:407] Connection header received
0925 11:01:01.759750 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.759757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.759855 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.759935 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:01.761635 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.762773 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.764537 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.765184 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.765284 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.770393 (+ 5109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:01.770409 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:01.770429 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:01.770471 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:01.774053 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:01.774770 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:01.774780 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:01.774783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:01.774914 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:01.775277 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:01.775282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:01.775284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:01.775659 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:01.775949 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.776248 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.776464 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:01:01.777446 29540 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:01.891273 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 29418
I20260925 11:01:01.904605 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 29551
Sep 25 11:01:01 dist-test-slave-trms krb5kdc[29313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:01:01.919248 29824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:01.905365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38966 (local address 127.24.159.194:42839)
0925 11:01:01.905556 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:01.905563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:01.905637 (+ 74us) server_negotiation.cc:407] Connection header received
0925 11:01:01.905812 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:01.905816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:01.905893 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:01.905977 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:01.907169 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.907971 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.909185 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:01.909618 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:01.909739 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:01.913562 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:01.913576 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:01.913595 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:01.913633 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:01.916536 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:01.917179 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:01.917187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:01.917190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:01.917355 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:01.917719 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:01.917725 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:01.917729 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:01.918074 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:01.918423 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:01.918666 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 11:01:01.918886 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260925 11:01:01.919899 29673 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:02.038152 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 29551
I20260925 11:01:02.052701 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 29683
Sep 25 11:01:02 dist-test-slave-trms krb5kdc[29313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:01:02.065984 29827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:02.053366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37196 (local address 127.24.159.195:38833)
0925 11:01:02.053521 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:02.053526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:02.053543 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:01:02.053686 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:02.053690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:02.053752 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:02.053846 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:02.055113 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:02.055835 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:02.057060 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:02.057445 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:02.057535 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:02.061455 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:02.061469 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:02.061485 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:02.061519 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:02.063907 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:02.064463 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:02.064469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:02.064471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:02.064598 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:02.064871 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:02.064874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:02.064876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:02.065182 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:02.065380 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:02.065581 (+ 201us) server_negotiation.cc:299] Negotiation successful
0925 11:01:02.065734 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 11:01:02.066622 29805 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:02.151916 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 29683
I20260925 11:01:02.165577 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 29326
I20260925 11:01:02.166656 29387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:02.274313 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 29326
2026-09-25T11:01:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3922 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-25T11:01:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:02Z Disabled control of system clock
I20260925 11:01:02.333151 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:35199
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:35199
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 11:01:02.642036 29863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:02.642321 29863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:02.642411 29863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:02.649326 29863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:02.649426 29863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:02.649467 29863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:02.649520 29863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:02.649566 29863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:02.658826 29863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:35199
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:35199
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:02.660099 29863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:02.661768 29863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:02.672688 29868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:02.672717 29871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:02.672770 29869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:02.673804 29863 server_base.cc:1034] running on GCE node
I20260925 11:01:02.674606 29863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:02.675915 29863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:02.677116 29863 hybrid_clock.cc:648] HybridClock initialized: now 1790334062677069 us; error 40 us; skew 500 ppm
I20260925 11:01:02.677426 29863 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:02.679587 29863 webserver.cc:460] Webserver started at http://127.24.159.254:37221/ using document root <none> and password file <none>
I20260925 11:01:02.680233 29863 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:02.680318 29863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:02.680572 29863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:02.683385 29863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "55a2690252d444259aa8d9baaa6f0775"
format_stamp: "Formatted at 2026-09-25 11:01:02 on dist-test-slave-trms"
server_key: "4b3e112f2844cc314b63929df3575fe2"
server_key_iv: "e2921174616788c9dee79ba4394280af"
server_key_version: "encryptionkey@0"
I20260925 11:01:02.684101 29863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "55a2690252d444259aa8d9baaa6f0775"
format_stamp: "Formatted at 2026-09-25 11:01:02 on dist-test-slave-trms"
server_key: "4b3e112f2844cc314b63929df3575fe2"
server_key_iv: "e2921174616788c9dee79ba4394280af"
server_key_version: "encryptionkey@0"
I20260925 11:01:02.688972 29863 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:01:02.692293 29877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:02.693578 29863 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:01:02.693734 29863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "55a2690252d444259aa8d9baaa6f0775"
format_stamp: "Formatted at 2026-09-25 11:01:02 on dist-test-slave-trms"
server_key: "4b3e112f2844cc314b63929df3575fe2"
server_key_iv: "e2921174616788c9dee79ba4394280af"
server_key_version: "encryptionkey@0"
I20260925 11:01:02.693931 29863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:02.699374 29863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:02.700117 29863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:02.700387 29863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:02.716957 29863 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:35199
I20260925 11:01:02.716988 29928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:35199 every 8 connection(s)
I20260925 11:01:02.718286 29863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:02.718688 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 29863
I20260925 11:01:02.718875 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:02.719635 25215 external_mini_cluster.cc:1428] Setting key 61143b05026ee61b6149b8b7d97d75c8
I20260925 11:01:02.722759 29929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:02.729127 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:02.721554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60824 (local address 127.24.159.254:35199)
0925 11:01:02.722635 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:02.722650 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:02.722702 (+ 52us) server_negotiation.cc:407] Connection header received
0925 11:01:02.724829 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:02.724855 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:02.725374 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:02.725819 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:02.726871 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:02.726880 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:02.726895 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:02.726920 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:02.727028 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:02.727052 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:02.727288 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:02.727415 (+ 127us) server_negotiation.cc:299] Negotiation successful
0925 11:01:02.727810 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":331,"threads_started":1}
I20260925 11:01:02.735954 29929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775: Bootstrap starting.
I20260925 11:01:02.739225 29929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:02.740321 29929 log.cc:824] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:02.743376 29929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775: No bootstrap required, opened a new log
I20260925 11:01:02.750687 29929 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2690252d444259aa8d9baaa6f0775" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35199 } }
I20260925 11:01:02.751067 29929 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:02.751156 29929 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55a2690252d444259aa8d9baaa6f0775, State: Initialized, Role: FOLLOWER
I20260925 11:01:02.751753 29929 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2690252d444259aa8d9baaa6f0775" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35199 } }
I20260925 11:01:02.751981 29929 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:02.752106 29929 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:02.752264 29929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:02.754431 29929 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2690252d444259aa8d9baaa6f0775" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35199 } }
I20260925 11:01:02.754931 29929 leader_election.cc:302] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55a2690252d444259aa8d9baaa6f0775; no voters:
I20260925 11:01:02.755509 29929 leader_election.cc:288] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:02.755627 29934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:02.756737 29934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [term 1 LEADER]: Becoming Leader. State: Replica: 55a2690252d444259aa8d9baaa6f0775, State: Running, Role: LEADER
I20260925 11:01:02.757290 29934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2690252d444259aa8d9baaa6f0775" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35199 } }
I20260925 11:01:02.757686 29929 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:02.759934 29936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55a2690252d444259aa8d9baaa6f0775. Latest consensus state: current_term: 1 leader_uuid: "55a2690252d444259aa8d9baaa6f0775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2690252d444259aa8d9baaa6f0775" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35199 } } }
I20260925 11:01:02.760339 29936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:02.760849 29935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55a2690252d444259aa8d9baaa6f0775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55a2690252d444259aa8d9baaa6f0775" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35199 } } }
I20260925 11:01:02.761176 29935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:02.761425 29943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:02.766975 29943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:02.775496 29943 catalog_manager.cc:1347] Generated new cluster ID: 33fed988bb2f48f4919497053113d836
I20260925 11:01:02.775617 29943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:02.795359 29943 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:02.795868 29943 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:02.802901 29943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775: Generated new TSK 0
I20260925 11:01:02.803897 29943 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:01:02.817723 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:35199
--builtin_ntp_servers=127.24.159.212:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 11:01:03.136638 29953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:03.136912 29953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:03.137037 29953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:03.144084 29953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:03.144179 29953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:03.144328 29953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:03.153147 29953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:35199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:03.154417 29953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:03.156102 29953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:03.168740 29959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:03.168778 29958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:03.168814 29961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:03.169400 29953 server_base.cc:1034] running on GCE node
I20260925 11:01:03.169919 29953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:03.170796 29953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:03.171976 29953 hybrid_clock.cc:648] HybridClock initialized: now 1790334063171929 us; error 46 us; skew 500 ppm
I20260925 11:01:03.172256 29953 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:03.174353 29953 webserver.cc:460] Webserver started at http://127.24.159.193:39313/ using document root <none> and password file <none>
I20260925 11:01:03.174989 29953 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:03.175079 29953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:03.175372 29953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:03.178037 29953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "df14d8ac459c4e239545c6b13c55388b"
format_stamp: "Formatted at 2026-09-25 11:01:03 on dist-test-slave-trms"
server_key: "679178fb1168eee7934adce492061a8f"
server_key_iv: "47fa16c160d553040817f4852bc9f5b0"
server_key_version: "encryptionkey@0"
I20260925 11:01:03.178812 29953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "df14d8ac459c4e239545c6b13c55388b"
format_stamp: "Formatted at 2026-09-25 11:01:03 on dist-test-slave-trms"
server_key: "679178fb1168eee7934adce492061a8f"
server_key_iv: "47fa16c160d553040817f4852bc9f5b0"
server_key_version: "encryptionkey@0"
I20260925 11:01:03.183473 29953 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260925 11:01:03.186604 29967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:03.187875 29953 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 11:01:03.188015 29953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "df14d8ac459c4e239545c6b13c55388b"
format_stamp: "Formatted at 2026-09-25 11:01:03 on dist-test-slave-trms"
server_key: "679178fb1168eee7934adce492061a8f"
server_key_iv: "47fa16c160d553040817f4852bc9f5b0"
server_key_version: "encryptionkey@0"
I20260925 11:01:03.188199 29953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:03.193630 29953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:03.194396 29953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:03.194634 29953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:03.195466 29953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:03.196759 29953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:03.196856 29953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:03.196959 29953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:03.197032 29953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:03.225003 29953 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:46135
I20260925 11:01:03.225045 30079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:46135 every 8 connection(s)
I20260925 11:01:03.226291 29953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:03.234131 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:03.228898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:32841 (local address 127.24.159.254:35199)
0925 11:01:03.229071 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:03.229076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:03.230452 (+ 1376us) server_negotiation.cc:407] Connection header received
0925 11:01:03.232096 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:03.232102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:03.232169 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:03.232253 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:03.233450 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:03.233456 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:03.233461 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:03.233478 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:03.233508 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:03.233520 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:03.233620 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:03.233796 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 11:01:03.233860 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:01:03.235028 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:03.229383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:35199 (local address 127.24.159.193:32841)
0925 11:01:03.230307 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:03.230350 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:03.231838 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:03.232498 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:03.232506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:03.232600 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:03.233171 (+ 571us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:03.233174 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:03.233210 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:03.233289 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:03.233605 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:03.233615 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:03.233622 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 11:01:03.233868 (+ 246us) client_negotiation.cc:240] Negotiation successful
0925 11:01:03.234114 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":254,"threads_started":1}
I20260925 11:01:03.236466 30080 heartbeater.cc:347] Connected to a master server at 127.24.159.254:35199
I20260925 11:01:03.236683 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 29953
I20260925 11:01:03.236730 30080 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:03.236851 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:01:03.237265 30080 heartbeater.cc:511] Master 127.24.159.254:35199 requested a full tablet report, sending...
I20260925 11:01:03.237470 25215 external_mini_cluster.cc:1428] Setting key 4dbb52d13b42c4cdb960f6ceb82c30a5
I20260925 11:01:03.239216 29894 ts_manager.cc:194] Registered new tserver with Master: df14d8ac459c4e239545c6b13c55388b (127.24.159.193:46135)
I20260925 11:01:03.240370 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:35199
--builtin_ntp_servers=127.24.159.212:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260925 11:01:03.546046 30083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:03.546357 30083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:03.546494 30083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:03.553903 30083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:03.554003 30083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:03.554139 30083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:03.563025 30083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:35199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:03.564316 30083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:03.565999 30083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:03.578256 30089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:03.578222 30088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:03.578465 30091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:03.579121 30083 server_base.cc:1034] running on GCE node
I20260925 11:01:03.579687 30083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:03.580592 30083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:03.581786 30083 hybrid_clock.cc:648] HybridClock initialized: now 1790334063581738 us; error 44 us; skew 500 ppm
I20260925 11:01:03.582094 30083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:03.584194 30083 webserver.cc:460] Webserver started at http://127.24.159.194:42539/ using document root <none> and password file <none>
I20260925 11:01:03.584896 30083 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:03.585005 30083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:03.585304 30083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:03.588094 30083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "bbf3250dd9194aacbf3d3e194a95713d"
format_stamp: "Formatted at 2026-09-25 11:01:03 on dist-test-slave-trms"
server_key: "86d5828c2b6238aba01003262cf88f1c"
server_key_iv: "74e27fe5c4539e45ec64328e8c2d25d7"
server_key_version: "encryptionkey@0"
I20260925 11:01:03.588871 30083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "bbf3250dd9194aacbf3d3e194a95713d"
format_stamp: "Formatted at 2026-09-25 11:01:03 on dist-test-slave-trms"
server_key: "86d5828c2b6238aba01003262cf88f1c"
server_key_iv: "74e27fe5c4539e45ec64328e8c2d25d7"
server_key_version: "encryptionkey@0"
I20260925 11:01:03.593907 30083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 11:01:03.597345 30097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:03.598749 30083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:03.598902 30083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "bbf3250dd9194aacbf3d3e194a95713d"
format_stamp: "Formatted at 2026-09-25 11:01:03 on dist-test-slave-trms"
server_key: "86d5828c2b6238aba01003262cf88f1c"
server_key_iv: "74e27fe5c4539e45ec64328e8c2d25d7"
server_key_version: "encryptionkey@0"
I20260925 11:01:03.599117 30083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:03.604668 30083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:03.605489 30083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:03.605739 30083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:03.606633 30083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:03.607954 30083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:03.608044 30083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:03.608150 30083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:03.608217 30083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:03.637061 30083 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:39685
I20260925 11:01:03.637110 30209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:39685 every 8 connection(s)
I20260925 11:01:03.638371 30083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:03.645642 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:03.640711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:34857 (local address 127.24.159.254:35199)
0925 11:01:03.640880 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:03.640884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:03.642219 (+ 1335us) server_negotiation.cc:407] Connection header received
0925 11:01:03.643820 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:03.643824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:03.643884 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:03.643969 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:03.645087 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:03.645091 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:03.645094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:03.645110 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:03.645141 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:03.645153 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:03.645238 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:03.645364 (+ 126us) server_negotiation.cc:299] Negotiation successful
0925 11:01:03.645411 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:01:03.645939 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 30083
I20260925 11:01:03.646097 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:01:03.646677 25215 external_mini_cluster.cc:1428] Setting key acffa8a6014812818a3a290c06d2a536
I20260925 11:01:03.646874 30212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:03.641175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:35199 (local address 127.24.159.194:34857)
0925 11:01:03.642054 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:03.642090 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:03.643596 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:03.644153 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:03.644162 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:03.644251 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:03.644820 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:03.644822 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:03.644859 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:03.644942 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:03.645220 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:03.645226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:03.645231 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 11:01:03.645423 (+ 192us) client_negotiation.cc:240] Negotiation successful
0925 11:01:03.645642 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":245,"threads_started":1}
I20260925 11:01:03.648272 30210 heartbeater.cc:347] Connected to a master server at 127.24.159.254:35199
I20260925 11:01:03.648572 30210 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:03.649139 30210 heartbeater.cc:511] Master 127.24.159.254:35199 requested a full tablet report, sending...
I20260925 11:01:03.649418 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:35199
--builtin_ntp_servers=127.24.159.212:40657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260925 11:01:03.650509 29893 ts_manager.cc:194] Registered new tserver with Master: bbf3250dd9194aacbf3d3e194a95713d (127.24.159.194:39685)
W20260925 11:01:03.961675 30213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:03.961939 30213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:03.962065 30213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:03.969336 30213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:03.969429 30213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:03.969568 30213 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:03.978463 30213 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:35199
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:03.979663 30213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:03.981355 30213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:03.993431 30218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:03.993556 30221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:03.993592 30219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:03.994340 30213 server_base.cc:1034] running on GCE node
I20260925 11:01:03.994952 30213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:03.995857 30213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:03.997054 30213 hybrid_clock.cc:648] HybridClock initialized: now 1790334063997004 us; error 45 us; skew 500 ppm
I20260925 11:01:03.997357 30213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:03.999612 30213 webserver.cc:460] Webserver started at http://127.24.159.195:36827/ using document root <none> and password file <none>
I20260925 11:01:04.000276 30213 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:04.000377 30213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:04.000692 30213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:04.003612 30213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d"
format_stamp: "Formatted at 2026-09-25 11:01:03 on dist-test-slave-trms"
server_key: "1bc93207ff03acfa140fae951f8e89f1"
server_key_iv: "736f735905d65e3e76763c1a0f858ac0"
server_key_version: "encryptionkey@0"
I20260925 11:01:04.004365 30213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d"
format_stamp: "Formatted at 2026-09-25 11:01:03 on dist-test-slave-trms"
server_key: "1bc93207ff03acfa140fae951f8e89f1"
server_key_iv: "736f735905d65e3e76763c1a0f858ac0"
server_key_version: "encryptionkey@0"
I20260925 11:01:04.009380 30213 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:01:04.012740 30227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:04.014055 30213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 11:01:04.014227 30213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d"
format_stamp: "Formatted at 2026-09-25 11:01:03 on dist-test-slave-trms"
server_key: "1bc93207ff03acfa140fae951f8e89f1"
server_key_iv: "736f735905d65e3e76763c1a0f858ac0"
server_key_version: "encryptionkey@0"
I20260925 11:01:04.014451 30213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:04.019882 30213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:04.020670 30213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:04.020918 30213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:04.021770 30213 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:04.023132 30213 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:04.023226 30213 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:04.023332 30213 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:04.023440 30213 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:04.053762 30213 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:43223
I20260925 11:01:04.053817 30339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:43223 every 8 connection(s)
I20260925 11:01:04.055083 30213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:04.055720 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 30213
I20260925 11:01:04.055902 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:01:04.056401 25215 external_mini_cluster.cc:1428] Setting key 31e3182dd52986d03e2584bf35a4a3db
I20260925 11:01:04.063403 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.057870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:58467 (local address 127.24.159.254:35199)
0925 11:01:04.058069 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.058076 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.059933 (+ 1857us) server_negotiation.cc:407] Connection header received
0925 11:01:04.061608 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.061611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.061663 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.061742 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.062878 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.062883 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.062886 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.062900 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.062930 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.062941 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.063035 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.063156 (+ 121us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.063201 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 11:01:04.064306 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.058648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:35199 (local address 127.24.159.195:58467)
0925 11:01:04.059783 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.059840 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:04.061388 (+ 1548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:04.061938 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:04.061946 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:04.062035 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:04.062617 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:04.062620 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:04.062656 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:04.062740 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:04.063011 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:04.063016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:04.063021 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 11:01:04.063214 (+ 193us) client_negotiation.cc:240] Negotiation successful
0925 11:01:04.063401 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":289,"threads_started":1}
I20260925 11:01:04.065524 30340 heartbeater.cc:347] Connected to a master server at 127.24.159.254:35199
I20260925 11:01:04.065786 30340 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:04.066355 30340 heartbeater.cc:511] Master 127.24.159.254:35199 requested a full tablet report, sending...
I20260925 11:01:04.067545 29893 ts_manager.cc:194] Registered new tserver with Master: 020c6a6d30cf49a1b22c9f2bdcf24c2d (127.24.159.195:43223)
I20260925 11:01:04.071451 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:01:04.077533 29932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.075443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60830 (local address 127.24.159.254:35199)
0925 11:01:04.075631 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.075638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.076023 (+ 385us) server_negotiation.cc:407] Connection header received
0925 11:01:04.076148 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.076151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.076197 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.076270 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.077027 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.077031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.077034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.077048 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.077077 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.077089 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.077176 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.077224 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.077264 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:01:04.080951 29893 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:01:04.087489 29893 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:04.103070 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.098866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39685 (local address 127.0.0.1:45232)
0925 11:01:04.099783 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.099821 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:04.099956 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:04.100823 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:04.100831 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:04.100925 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:04.101652 (+ 727us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:04.101656 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:04.101685 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:04.101770 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:04.102333 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:04.102345 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:04.102353 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 11:01:04.102521 (+ 168us) client_negotiation.cc:240] Negotiation successful
0925 11:01:04.102610 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":503,"threads_started":1}
I20260925 11:01:04.103932 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.099014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.24.159.194:39685)
0925 11:01:04.100391 (+ 1377us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.100398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.100422 (+ 24us) server_negotiation.cc:407] Connection header received
0925 11:01:04.100484 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.100489 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.100659 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.100791 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.102069 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.102078 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.102091 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.102115 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.102170 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.102217 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.103547 (+ 1330us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.103606 (+ 59us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.103680 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1217,"thread_start_us":1095,"threads_started":1}
I20260925 11:01:04.107793 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.100248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46135 (local address 127.0.0.1:58534)
0925 11:01:04.103255 (+ 3007us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.103271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:04.103339 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:04.104825 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:04.104830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:04.104873 (+ 43us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:04.106033 (+ 1160us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:04.106037 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:04.106053 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:04.106094 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:04.107408 (+ 1314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:04.107412 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:04.107417 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 11:01:04.107498 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 11:01:04.107546 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2925,"thread_start_us":4312,"threads_started":1}
I20260925 11:01:04.108042 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.100586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58534 (local address 127.24.159.193:46135)
0925 11:01:04.104440 (+ 3854us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.104446 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.104469 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:01:04.104520 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.104526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.104681 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.104806 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.106258 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.106267 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.106281 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.106307 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.107255 (+ 948us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.107284 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.107443 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.107520 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.107610 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3735,"thread_start_us":3626,"threads_started":1}
I20260925 11:01:04.108274 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.104857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:43223 (local address 127.0.0.1:52460)
0925 11:01:04.105844 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.105860 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:04.105959 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:04.106515 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:04.106520 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:04.106542 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:04.107168 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:04.107171 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:04.107186 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:04.107228 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:04.107775 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:04.107784 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:04.107789 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 11:01:04.107919 (+ 130us) client_negotiation.cc:240] Negotiation successful
0925 11:01:04.107966 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856}
I20260925 11:01:04.108948 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.105172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52460 (local address 127.24.159.195:43223)
0925 11:01:04.105826 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.105834 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.105952 (+ 118us) server_negotiation.cc:407] Connection header received
0925 11:01:04.106116 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.106123 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.106353 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.106505 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.107468 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.107476 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.107487 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.107503 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.107549 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.107565 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.107719 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.108481 (+ 762us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.108565 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":355,"threads_started":1}
I20260925 11:01:04.109129 30145 tablet_service.cc:1467] Processing CreateTablet for tablet 860544234ad24eb58e327626bfcf50b3 (DEFAULT_TABLE table=test-table [id=fe2120dfe6354011b3f553ef824a0169]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:04.110550 30145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 860544234ad24eb58e327626bfcf50b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:04.112497 30275 tablet_service.cc:1467] Processing CreateTablet for tablet 860544234ad24eb58e327626bfcf50b3 (DEFAULT_TABLE table=test-table [id=fe2120dfe6354011b3f553ef824a0169]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:04.112705 30015 tablet_service.cc:1467] Processing CreateTablet for tablet 860544234ad24eb58e327626bfcf50b3 (DEFAULT_TABLE table=test-table [id=fe2120dfe6354011b3f553ef824a0169]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:04.113561 30275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 860544234ad24eb58e327626bfcf50b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:04.113977 30015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 860544234ad24eb58e327626bfcf50b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:04.124493 30357 tablet_bootstrap.cc:492] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: Bootstrap starting.
I20260925 11:01:04.126861 30358 tablet_bootstrap.cc:492] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: Bootstrap starting.
I20260925 11:01:04.128281 30359 tablet_bootstrap.cc:492] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b: Bootstrap starting.
I20260925 11:01:04.128961 30357 tablet_bootstrap.cc:654] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:04.130319 30358 tablet_bootstrap.cc:654] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:04.130548 30357 log.cc:824] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:04.131327 30359 tablet_bootstrap.cc:654] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:04.131448 30358 log.cc:824] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:04.132310 30359 log.cc:824] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:04.133813 30357 tablet_bootstrap.cc:492] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: No bootstrap required, opened a new log
I20260925 11:01:04.134117 30357 ts_tablet_manager.cc:1397] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260925 11:01:04.134632 30359 tablet_bootstrap.cc:492] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b: No bootstrap required, opened a new log
I20260925 11:01:04.134855 30359 ts_tablet_manager.cc:1397] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260925 11:01:04.141362 30358 tablet_bootstrap.cc:492] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: No bootstrap required, opened a new log
I20260925 11:01:04.141639 30358 ts_tablet_manager.cc:1397] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260925 11:01:04.142421 30359 raft_consensus.cc:348] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } }
I20260925 11:01:04.142769 30359 raft_consensus.cc:374] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:04.142856 30359 raft_consensus.cc:729] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df14d8ac459c4e239545c6b13c55388b, State: Initialized, Role: FOLLOWER
I20260925 11:01:04.143424 30359 consensus_queue.cc:260] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } }
I20260925 11:01:04.143975 30357 raft_consensus.cc:348] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } }
I20260925 11:01:04.144394 30357 raft_consensus.cc:374] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:04.144420 30080 heartbeater.cc:503] Master 127.24.159.254:35199 was elected leader, sending a full tablet report...
I20260925 11:01:04.144484 30357 raft_consensus.cc:729] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 020c6a6d30cf49a1b22c9f2bdcf24c2d, State: Initialized, Role: FOLLOWER
I20260925 11:01:04.145100 30359 ts_tablet_manager.cc:1428] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260925 11:01:04.145141 30357 consensus_queue.cc:260] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } }
I20260925 11:01:04.146884 30340 heartbeater.cc:503] Master 127.24.159.254:35199 was elected leader, sending a full tablet report...
I20260925 11:01:04.147505 30357 ts_tablet_manager.cc:1428] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: Time spent starting tablet: real 0.013s user 0.006s sys 0.006s
I20260925 11:01:04.151171 30358 raft_consensus.cc:348] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } }
I20260925 11:01:04.151623 30358 raft_consensus.cc:374] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:04.151723 30358 raft_consensus.cc:729] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbf3250dd9194aacbf3d3e194a95713d, State: Initialized, Role: FOLLOWER
I20260925 11:01:04.152237 30358 consensus_queue.cc:260] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } }
I20260925 11:01:04.153112 30210 heartbeater.cc:503] Master 127.24.159.254:35199 was elected leader, sending a full tablet report...
I20260925 11:01:04.153787 30358 ts_tablet_manager.cc:1428] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
W20260925 11:01:04.228287 30081 tablet.cc:2369] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:01:04.307005 30341 tablet.cc:2369] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:01:04.312561 30364 raft_consensus.cc:482] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:04.312830 30364 raft_consensus.cc:504] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } }
I20260925 11:01:04.314497 30364 leader_election.cc:288] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbf3250dd9194aacbf3d3e194a95713d (127.24.159.194:39685), df14d8ac459c4e239545c6b13c55388b (127.24.159.193:46135)
I20260925 11:01:04.316982 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.314866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:53307 (local address 127.24.159.194:39685)
0925 11:01:04.315025 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.315031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.315051 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:01:04.315252 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.315263 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.315326 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.315433 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.316406 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.316410 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.316413 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.316430 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.316463 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.316474 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.316562 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.316676 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.316730 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 11:01:04.316978 30342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.314795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39685 (local address 127.24.159.195:53307)
0925 11:01:04.314964 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.314987 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:04.315096 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:04.315605 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:04.315613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:04.315638 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:04.316217 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:04.316221 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:04.316237 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:04.316282 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:04.316551 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:04.316554 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:04.316558 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 11:01:04.316652 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 11:01:04.316726 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 11:01:04.318010 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.315467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:47519 (local address 127.24.159.193:46135)
0925 11:01:04.315759 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.315765 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.315806 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:01:04.316211 (+ 405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.316217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.316277 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.316376 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.317463 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.317469 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.317474 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.317490 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.317522 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.317537 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.317647 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.317698 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.317755 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260925 11:01:04.318054 30366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.315187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46135 (local address 127.24.159.195:47519)
0925 11:01:04.315731 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.315748 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:04.315842 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:04.316623 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:04.316626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:04.316642 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:04.317269 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:04.317272 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:04.317287 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:04.317328 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:04.317619 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:04.317622 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:04.317626 (+ 4us) client_negotiation.cc:769] Sending connection context
0925 11:01:04.317714 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 11:01:04.317751 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":267,"threads_started":1}
I20260925 11:01:04.318032 30165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "860544234ad24eb58e327626bfcf50b3" candidate_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbf3250dd9194aacbf3d3e194a95713d" is_pre_election: true
I20260925 11:01:04.318581 30165 raft_consensus.cc:2454] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 020c6a6d30cf49a1b22c9f2bdcf24c2d in term 0.
I20260925 11:01:04.319197 30230 leader_election.cc:302] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 020c6a6d30cf49a1b22c9f2bdcf24c2d, bbf3250dd9194aacbf3d3e194a95713d; no voters:
I20260925 11:01:04.319074 30035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "860544234ad24eb58e327626bfcf50b3" candidate_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df14d8ac459c4e239545c6b13c55388b" is_pre_election: true
I20260925 11:01:04.319583 30364 raft_consensus.cc:2781] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:04.319612 30035 raft_consensus.cc:2454] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 020c6a6d30cf49a1b22c9f2bdcf24c2d in term 0.
I20260925 11:01:04.319712 30364 raft_consensus.cc:482] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:04.319808 30364 raft_consensus.cc:3037] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:04.321924 30364 raft_consensus.cc:504] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } }
I20260925 11:01:04.322522 30364 leader_election.cc:288] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [CANDIDATE]: Term 1 election: Requested vote from peers bbf3250dd9194aacbf3d3e194a95713d (127.24.159.194:39685), df14d8ac459c4e239545c6b13c55388b (127.24.159.193:46135)
I20260925 11:01:04.323001 30165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "860544234ad24eb58e327626bfcf50b3" candidate_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbf3250dd9194aacbf3d3e194a95713d"
I20260925 11:01:04.323001 30035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "860544234ad24eb58e327626bfcf50b3" candidate_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df14d8ac459c4e239545c6b13c55388b"
I20260925 11:01:04.323211 30035 raft_consensus.cc:3037] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:04.323211 30165 raft_consensus.cc:3037] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:04.326298 30165 raft_consensus.cc:2454] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 020c6a6d30cf49a1b22c9f2bdcf24c2d in term 1.
I20260925 11:01:04.326298 30035 raft_consensus.cc:2454] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 020c6a6d30cf49a1b22c9f2bdcf24c2d in term 1.
I20260925 11:01:04.326664 30230 leader_election.cc:302] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 020c6a6d30cf49a1b22c9f2bdcf24c2d, df14d8ac459c4e239545c6b13c55388b; no voters:
I20260925 11:01:04.326910 30364 raft_consensus.cc:2781] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:04.327314 30364 raft_consensus.cc:686] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 1 LEADER]: Becoming Leader. State: Replica: 020c6a6d30cf49a1b22c9f2bdcf24c2d, State: Running, Role: LEADER
I20260925 11:01:04.327818 30364 consensus_queue.cc:237] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } }
I20260925 11:01:04.332515 29894 catalog_manager.cc:5495] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 020c6a6d30cf49a1b22c9f2bdcf24c2d (127.24.159.195). New cstate: current_term: 1 leader_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "020c6a6d30cf49a1b22c9f2bdcf24c2d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43223 } health_report { overall_health: HEALTHY } } }
W20260925 11:01:04.390235 30211 tablet.cc:2369] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:01:04.400628 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.398539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52462 (local address 127.24.159.195:43223)
0925 11:01:04.398724 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.398730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.398810 (+ 80us) server_negotiation.cc:407] Connection header received
0925 11:01:04.399043 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.399047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.399126 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.399208 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.400056 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.400062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.400066 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.400082 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.400114 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.400126 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.400230 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.400317 (+ 87us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.400375 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 11:01:04.408571 30165 raft_consensus.cc:1261] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [term 1 FOLLOWER]: Refusing update from remote peer 020c6a6d30cf49a1b22c9f2bdcf24c2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:04.408571 30035 raft_consensus.cc:1261] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b [term 1 FOLLOWER]: Refusing update from remote peer 020c6a6d30cf49a1b22c9f2bdcf24c2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:04.409541 30367 consensus_queue.cc:1035] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbf3250dd9194aacbf3d3e194a95713d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:04.409885 30364 consensus_queue.cc:1035] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "df14d8ac459c4e239545c6b13c55388b" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 46135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:04.433540 29894 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:04.433769 29894 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:04.437276 29894 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 55a2690252d444259aa8d9baaa6f0775: Sending DeleteTablet for 3 replicas of tablet 860544234ad24eb58e327626bfcf50b3
I20260925 11:01:04.438444 30145 tablet_service.cc:1514] Processing DeleteTablet for tablet 860544234ad24eb58e327626bfcf50b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:04 UTC) from {username='slave'} at 127.0.0.1:45232
I20260925 11:01:04.438988 30275 tablet_service.cc:1514] Processing DeleteTablet for tablet 860544234ad24eb58e327626bfcf50b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:04 UTC) from {username='slave'} at 127.0.0.1:52460
I20260925 11:01:04.439620 30380 tablet_replica.cc:330] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: stopping tablet replica
I20260925 11:01:04.439636 30015 tablet_service.cc:1514] Processing DeleteTablet for tablet 860544234ad24eb58e327626bfcf50b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:04 UTC) from {username='slave'} at 127.0.0.1:58534
I20260925 11:01:04.439961 30380 raft_consensus.cc:2229] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:04.440279 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 29953
I20260925 11:01:04.440454 30380 raft_consensus.cc:2258] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:04.444043 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.441087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58550 (local address 127.24.159.193:46135)
0925 11:01:04.441249 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.441255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.441683 (+ 428us) server_negotiation.cc:407] Connection header received
0925 11:01:04.441941 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.441947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.442049 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.442127 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.443269 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.443275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.443279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.443298 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.443336 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.443351 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.443455 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.443684 (+ 229us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.443744 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 11:01:04.444883 30380 ts_tablet_manager.cc:1905] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:04.445037 30075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:04.449572 30380 ts_tablet_manager.cc:1918] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:04.449708 30380 log.cc:1196] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/860544234ad24eb58e327626bfcf50b3
I20260925 11:01:04.450167 30380 ts_tablet_manager.cc:1939] T 860544234ad24eb58e327626bfcf50b3 P bbf3250dd9194aacbf3d3e194a95713d: Deleting consensus metadata
I20260925 11:01:04.450913 30381 tablet_replica.cc:330] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: stopping tablet replica
I20260925 11:01:04.452096 29880 catalog_manager.cc:4841] TS bbf3250dd9194aacbf3d3e194a95713d (127.24.159.194:39685): tablet 860544234ad24eb58e327626bfcf50b3 (table test-table [id=fe2120dfe6354011b3f553ef824a0169]) successfully deleted
I20260925 11:01:04.452258 30381 raft_consensus.cc:2229] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:01:04.453001 30381 raft_consensus.cc:2258] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:04.455243 30381 ts_tablet_manager.cc:1905] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:04.462549 30381 ts_tablet_manager.cc:1918] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:04.462728 30381 log.cc:1196] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/860544234ad24eb58e327626bfcf50b3
I20260925 11:01:04.463280 30381 ts_tablet_manager.cc:1939] T 860544234ad24eb58e327626bfcf50b3 P 020c6a6d30cf49a1b22c9f2bdcf24c2d: Deleting consensus metadata
I20260925 11:01:04.465165 29880 catalog_manager.cc:4841] TS 020c6a6d30cf49a1b22c9f2bdcf24c2d (127.24.159.195:43223): tablet 860544234ad24eb58e327626bfcf50b3 (table test-table [id=fe2120dfe6354011b3f553ef824a0169]) successfully deleted
I20260925 11:01:04.540333 30382 tablet_replica.cc:330] T 860544234ad24eb58e327626bfcf50b3 P df14d8ac459c4e239545c6b13c55388b: stopping tablet replica
I20260925 11:01:04.540566 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 29953
W20260925 11:01:04.554045 29880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.159.193:46135 (error 108)
I20260925 11:01:04.554869 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 30083
I20260925 11:01:04.556268 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.555610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46135 (local address 127.0.0.1:58554)
0925 11:01:04.555913 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.556093 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:46135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
W20260925 11:01:04.556522 29880 catalog_manager.cc:4584] TS df14d8ac459c4e239545c6b13c55388b (127.24.159.193:46135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 860544234ad24eb58e327626bfcf50b3: Network error: Client connection negotiation failed: client connection to 127.24.159.193:46135: connect: Connection refused (error 111)
I20260925 11:01:04.557819 30353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.555747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.24.159.194:39685)
0925 11:01:04.555911 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.555917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.555970 (+ 53us) server_negotiation.cc:407] Connection header received
0925 11:01:04.556166 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.556172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.556264 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.556347 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.557208 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.557214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.557218 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.557235 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.557265 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.557277 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.557389 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.557486 (+ 97us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.557547 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:01:04.558585 30205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:04.610391 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.609799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46135 (local address 127.0.0.1:58556)
0925 11:01:04.610043 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.610167 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:46135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260925 11:01:04.611605 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.611207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46135 (local address 127.0.0.1:58558)
0925 11:01:04.611362 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.611454 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:46135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 11:01:04.644984 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 30083
I20260925 11:01:04.664815 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 30213
I20260925 11:01:04.667673 30356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.665556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52468 (local address 127.24.159.195:43223)
0925 11:01:04.665727 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:04.665731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:04.665808 (+ 77us) server_negotiation.cc:407] Connection header received
0925 11:01:04.666020 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:04.666024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:04.666075 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:04.666154 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:04.667047 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:04.667051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:04.667054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:04.667068 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:04.667096 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:04.667108 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:04.667220 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:04.667321 (+ 101us) server_negotiation.cc:299] Negotiation successful
0925 11:01:04.667368 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 11:01:04.668322 30335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:04.672684 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.672140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46135 (local address 127.0.0.1:58560)
0925 11:01:04.672399 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.672492 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:46135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260925 11:01:04.673610 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.673318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46135 (local address 127.0.0.1:58570)
0925 11:01:04.673439 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.673504 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:46135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 11:01:04.754382 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.753801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46135 (local address 127.0.0.1:58586)
0925 11:01:04.754040 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.754136 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:46135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260925 11:01:04.755390 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:04.755017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:46135 (local address 127.0.0.1:58590)
0925 11:01:04.755201 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:04.755278 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:46135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260925 11:01:04.782159 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 30213
I20260925 11:01:04.797768 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 29863
I20260925 11:01:04.798934 29924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:04.901445 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 29863
2026-09-25T11:01:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2626 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260925 11:01:04.937413 25215 mini_oidc.cc:152] Starting JWKS server
I20260925 11:01:04.940021 25215 webserver.cc:460] Webserver started at https://127.0.0.1:45433/ using document root <none> and password file <none>
I20260925 11:01:04.940305 25215 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 11:01:04.940891 25215 webserver.cc:460] Webserver started at http://127.0.0.1:46635/ using document root <none> and password file <none>
2026-09-25T11:01:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:04Z Disabled control of system clock
I20260925 11:01:04.963797 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:46011
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:36759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:46011
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45433/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/testchainca.cert with env {}
W20260925 11:01:05.276777 30394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:05.277072 30394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:05.277197 30394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:05.284288 30394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:05.284400 30394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:05.284462 30394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:05.284539 30394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:05.284595 30394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 11:01:05.284677 30394 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 11:01:05.284726 30394 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45433/jwks.json
I20260925 11:01:05.294363 30394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:46011
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:46011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45433/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:05.295595 30394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:05.296841 30394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:05.307951 30399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:05.308002 30400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:05.307981 30402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:05.309154 30394 server_base.cc:1034] running on GCE node
I20260925 11:01:05.309908 30394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:05.311302 30394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:05.312495 30394 hybrid_clock.cc:648] HybridClock initialized: now 1790334065312456 us; error 34 us; skew 500 ppm
I20260925 11:01:05.312821 30394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:05.314925 30394 webserver.cc:460] Webserver started at http://127.24.159.254:43749/ using document root <none> and password file <none>
I20260925 11:01:05.315554 30394 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:05.315647 30394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:05.315944 30394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:05.318743 30394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "508587f9e5a04c02bfa1aebbd5d6e497"
format_stamp: "Formatted at 2026-09-25 11:01:05 on dist-test-slave-trms"
server_key: "a50d7416b0b2680dea4e8aabbb9292ca"
server_key_iv: "eedd135fceed7285a87d457294792e5c"
server_key_version: "encryptionkey@0"
I20260925 11:01:05.319476 30394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "508587f9e5a04c02bfa1aebbd5d6e497"
format_stamp: "Formatted at 2026-09-25 11:01:05 on dist-test-slave-trms"
server_key: "a50d7416b0b2680dea4e8aabbb9292ca"
server_key_iv: "eedd135fceed7285a87d457294792e5c"
server_key_version: "encryptionkey@0"
I20260925 11:01:05.324237 30394 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 11:01:05.327453 30408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:05.328717 30394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:01:05.328884 30394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "508587f9e5a04c02bfa1aebbd5d6e497"
format_stamp: "Formatted at 2026-09-25 11:01:05 on dist-test-slave-trms"
server_key: "a50d7416b0b2680dea4e8aabbb9292ca"
server_key_iv: "eedd135fceed7285a87d457294792e5c"
server_key_version: "encryptionkey@0"
I20260925 11:01:05.329071 30394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:05.341733 30394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:05.342592 30394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:05.342866 30394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:05.359298 30394 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:46011
I20260925 11:01:05.359318 30459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:46011 every 8 connection(s)
I20260925 11:01:05.360577 30394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:05.364243 30460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:05.368573 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 30394
I20260925 11:01:05.368809 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:05.369537 25215 external_mini_cluster.cc:1428] Setting key 8f275e3c9a984227c064a08191b8b8e0
I20260925 11:01:05.375700 30460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497: Bootstrap starting.
I20260925 11:01:05.378791 30460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:05.379786 30460 log.cc:824] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:05.383101 30460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497: No bootstrap required, opened a new log
I20260925 11:01:05.383199 30463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:05.371500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35666 (local address 127.24.159.254:46011)
0925 11:01:05.372487 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:05.372499 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:05.372544 (+ 45us) server_negotiation.cc:407] Connection header received
0925 11:01:05.374365 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:05.374398 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:05.374879 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:05.375244 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:05.375811 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:05.377119 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:05.378480 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:05.379199 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:05.379388 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:05.380565 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:05.380573 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:05.380587 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:05.381159 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:05.381259 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:05.381286 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:05.381408 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:05.381600 (+ 192us) server_negotiation.cc:299] Negotiation successful
0925 11:01:05.381929 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":325,"threads_started":1}
I20260925 11:01:05.387064 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:01:05.392328 30460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508587f9e5a04c02bfa1aebbd5d6e497" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46011 } }
I20260925 11:01:05.392649 30460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:05.392722 30460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508587f9e5a04c02bfa1aebbd5d6e497, State: Initialized, Role: FOLLOWER
I20260925 11:01:05.393201 30460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508587f9e5a04c02bfa1aebbd5d6e497" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46011 } }
I20260925 11:01:05.393383 30460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:05.393478 30460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:05.393602 30460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:05.395710 30460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508587f9e5a04c02bfa1aebbd5d6e497" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46011 } }
I20260925 11:01:05.396155 30460 leader_election.cc:302] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 508587f9e5a04c02bfa1aebbd5d6e497; no voters:
I20260925 11:01:05.396664 30460 leader_election.cc:288] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:05.396850 30465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:05.397785 30465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [term 1 LEADER]: Becoming Leader. State: Replica: 508587f9e5a04c02bfa1aebbd5d6e497, State: Running, Role: LEADER
I20260925 11:01:05.398296 30465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508587f9e5a04c02bfa1aebbd5d6e497" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46011 } }
I20260925 11:01:05.398901 30460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:05.400955 30467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 508587f9e5a04c02bfa1aebbd5d6e497. Latest consensus state: current_term: 1 leader_uuid: "508587f9e5a04c02bfa1aebbd5d6e497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508587f9e5a04c02bfa1aebbd5d6e497" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46011 } } }
I20260925 11:01:05.401379 30467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:05.401518 30466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "508587f9e5a04c02bfa1aebbd5d6e497" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "508587f9e5a04c02bfa1aebbd5d6e497" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46011 } } }
I20260925 11:01:05.402803 30466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:05.403875 30474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:05.408825 30474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:05.417990 30474 catalog_manager.cc:1347] Generated new cluster ID: f7ef4ab5f366465fb3cd2b742c59bf8d
I20260925 11:01:05.418118 30474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:05.430405 30474 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:05.431694 30474 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:05.439905 30474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 508587f9e5a04c02bfa1aebbd5d6e497: Generated new TSK 0
I20260925 11:01:05.440843 30474 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 11:01:05.475749 30490 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260925 11:01:05.475899 30490 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 11:01:05.477156 30463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:05.455889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35676 (local address 127.24.159.254:46011)
0925 11:01:05.456089 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:05.456096 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:05.456531 (+ 435us) server_negotiation.cc:407] Connection header received
0925 11:01:05.456800 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:05.456806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:05.456915 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:05.457079 (+ 164us) server_negotiation.cc:226] Negotiated authn=JWT
0925 11:01:05.457565 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:05.458689 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:05.460058 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:05.460717 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:05.460846 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:05.461362 (+ 516us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 11:01:05.476444 (+ 15082us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 11:01:05.476836 (+ 392us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35676: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260925 11:01:05.477278 30463 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35676: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260925 11:01:05.478696 30489 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:05.455752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:46011 (local address 127.0.0.1:35676)
0925 11:01:05.456425 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:05.456441 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:05.456584 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:05.457093 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:05.457096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:05.457106 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0925 11:01:05.457409 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:05.457417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:05.458817 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:05.458820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:05.459909 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:05.459919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:05.460876 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:05.460880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:05.461015 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:05.461076 (+ 61us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 11:01:05.476752 (+ 15676us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0925 11:01:05.477046 (+ 294us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.159.254:46011: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":367,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.159.254:46011: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45433/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260925 11:01:05.480568 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 30394
I20260925 11:01:05.481684 30455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:05.573632 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 30394
2026-09-25T11:01:05Z chronyd exiting
I20260925 11:01:05.620158 25215 test_util.cc:175] -----------------------------------------------
I20260925 11:01:05.620291 25215 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790334022505657-25215-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (687 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260925 11:01:05.623845 25215 mini_oidc.cc:152] Starting JWKS server
I20260925 11:01:05.625567 25215 webserver.cc:460] Webserver started at https://127.0.0.1:42011/ using document root <none> and password file <none>
I20260925 11:01:05.625763 25215 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 11:01:05.626402 25215 webserver.cc:460] Webserver started at http://127.0.0.1:35105/ using document root <none> and password file <none>
2026-09-25T11:01:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:05Z Disabled control of system clock
I20260925 11:01:05.648393 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:42563
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:38215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:42563
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42011/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/testchainca.cert with env {}
W20260925 11:01:05.953792 30497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:05.954094 30497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:05.954241 30497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:05.961339 30497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:05.961442 30497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:05.961505 30497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:05.961576 30497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:05.961634 30497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 11:01:05.961715 30497 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 11:01:05.961760 30497 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42011/jwks.json
I20260925 11:01:05.971556 30497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:42563
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:42563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42011/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:05.972815 30497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:05.974090 30497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:05.984987 30503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:05.985036 30505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:05.985133 30502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:05.985855 30497 server_base.cc:1034] running on GCE node
I20260925 11:01:05.986671 30497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:05.988049 30497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:05.989235 30497 hybrid_clock.cc:648] HybridClock initialized: now 1790334065989197 us; error 30 us; skew 500 ppm
I20260925 11:01:05.989562 30497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:05.991748 30497 webserver.cc:460] Webserver started at http://127.24.159.254:42981/ using document root <none> and password file <none>
I20260925 11:01:05.992424 30497 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:05.992519 30497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:05.992831 30497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:05.996811 30497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "7cd0324c274443b8b1c58721f60e994e"
format_stamp: "Formatted at 2026-09-25 11:01:05 on dist-test-slave-trms"
server_key: "9dcf0c58f34c70675716d3cbb2bfeecb"
server_key_iv: "965f4d98c847817fe1155a8e62cd892a"
server_key_version: "encryptionkey@0"
I20260925 11:01:05.997591 30497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "7cd0324c274443b8b1c58721f60e994e"
format_stamp: "Formatted at 2026-09-25 11:01:05 on dist-test-slave-trms"
server_key: "9dcf0c58f34c70675716d3cbb2bfeecb"
server_key_iv: "965f4d98c847817fe1155a8e62cd892a"
server_key_version: "encryptionkey@0"
I20260925 11:01:06.002580 30497 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 11:01:06.005792 30511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:06.007193 30497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:06.007359 30497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "7cd0324c274443b8b1c58721f60e994e"
format_stamp: "Formatted at 2026-09-25 11:01:05 on dist-test-slave-trms"
server_key: "9dcf0c58f34c70675716d3cbb2bfeecb"
server_key_iv: "965f4d98c847817fe1155a8e62cd892a"
server_key_version: "encryptionkey@0"
I20260925 11:01:06.007556 30497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:06.020107 30497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:06.020848 30497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:06.021113 30497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:06.047962 30497 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:42563
I20260925 11:01:06.048023 30562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:42563 every 8 connection(s)
I20260925 11:01:06.049814 30497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:06.052700 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 30497
I20260925 11:01:06.052896 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:06.053663 25215 external_mini_cluster.cc:1428] Setting key b7e52672d9665a4d7d3cf9e19895c4e1
I20260925 11:01:06.054828 30563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:06.068311 30563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e: Bootstrap starting.
I20260925 11:01:06.069828 30566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:06.055744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35650 (local address 127.24.159.254:42563)
0925 11:01:06.056925 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:06.056946 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:06.058804 (+ 1858us) server_negotiation.cc:407] Connection header received
0925 11:01:06.061101 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:06.061126 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:06.061637 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:06.061990 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:06.062514 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.063734 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.065490 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.066246 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.066381 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:06.067497 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:06.067505 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:06.067519 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:06.068066 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:06.068145 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:06.068166 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:06.068251 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:06.068500 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 11:01:06.068866 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":365,"threads_started":1}
I20260925 11:01:06.071635 30563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:06.072757 30563 log.cc:824] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:06.073388 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:01:06.075433 30563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e: No bootstrap required, opened a new log
I20260925 11:01:06.082242 30563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd0324c274443b8b1c58721f60e994e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42563 } }
I20260925 11:01:06.082593 30563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:06.082692 30563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd0324c274443b8b1c58721f60e994e, State: Initialized, Role: FOLLOWER
I20260925 11:01:06.083324 30563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd0324c274443b8b1c58721f60e994e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42563 } }
I20260925 11:01:06.083587 30563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:06.083691 30563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:06.083810 30563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:06.085804 30563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd0324c274443b8b1c58721f60e994e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42563 } }
I20260925 11:01:06.086295 30563 leader_election.cc:302] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cd0324c274443b8b1c58721f60e994e; no voters:
I20260925 11:01:06.086840 30563 leader_election.cc:288] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:06.086988 30568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:06.087605 30568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [term 1 LEADER]: Becoming Leader. State: Replica: 7cd0324c274443b8b1c58721f60e994e, State: Running, Role: LEADER
I20260925 11:01:06.088078 30568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd0324c274443b8b1c58721f60e994e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42563 } }
I20260925 11:01:06.088806 30563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:06.090845 30569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7cd0324c274443b8b1c58721f60e994e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd0324c274443b8b1c58721f60e994e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42563 } } }
I20260925 11:01:06.091321 30569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:06.091687 30570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7cd0324c274443b8b1c58721f60e994e. Latest consensus state: current_term: 1 leader_uuid: "7cd0324c274443b8b1c58721f60e994e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cd0324c274443b8b1c58721f60e994e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42563 } } }
I20260925 11:01:06.091995 30570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:06.093961 30573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:06.099437 30573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:06.108750 30573 catalog_manager.cc:1347] Generated new cluster ID: 6619a98f898449d497390de72000636a
I20260925 11:01:06.108868 30573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:06.130115 30573 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:06.131412 30573 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:06.137930 30573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7cd0324c274443b8b1c58721f60e994e: Generated new TSK 0
I20260925 11:01:06.138799 30573 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260925 11:01:06.177307 30593 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260925 11:01:06.177440 30593 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 11:01:06.178323 30566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:06.151530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.24.159.254:42563)
0925 11:01:06.151699 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:06.151706 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:06.152196 (+ 490us) server_negotiation.cc:407] Connection header received
0925 11:01:06.152421 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:06.152425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:06.152490 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:06.152609 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0925 11:01:06.153115 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.153801 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.155792 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.156215 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.156302 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:06.156813 (+ 511us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 11:01:06.177759 (+ 20946us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 11:01:06.178069 (+ 310us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35654: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":40}
W20260925 11:01:06.178438 30566 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35654: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260925 11:01:06.178653 30592 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:06.151309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:42563 (local address 127.0.0.1:35654)
0925 11:01:06.152089 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:06.152108 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:06.152257 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:06.152648 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:06.152654 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:06.152668 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0925 11:01:06.152979 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:06.152991 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.153946 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.153953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:06.155658 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:06.155674 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.156363 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.156370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:06.156544 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:06.156587 (+ 43us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 11:01:06.177991 (+ 21404us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0925 11:01:06.178274 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.159.254:42563: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42011/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":485,"threads_started":1}
I20260925 11:01:06.179695 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 30497
I20260925 11:01:06.180862 30558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:06.271867 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 30497
2026-09-25T11:01:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (699 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260925 11:01:06.323374 25215 mini_oidc.cc:152] Starting JWKS server
I20260925 11:01:06.325129 25215 webserver.cc:460] Webserver started at https://127.0.0.1:43917/ using document root <none> and password file <none>
I20260925 11:01:06.325348 25215 mini_oidc.cc:189] Starting OIDC Discovery server
I20260925 11:01:06.325928 25215 webserver.cc:460] Webserver started at http://127.0.0.1:43637/ using document root <none> and password file <none>
2026-09-25T11:01:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:06Z Disabled control of system clock
I20260925 11:01:06.348230 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:38587
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:40961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:38587
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43917/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 11:01:06.655200 30600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:06.655496 30600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:06.655632 30600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:06.662735 30600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:06.662847 30600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:06.662911 30600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:06.662992 30600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:06.663053 30600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 11:01:06.663136 30600 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 11:01:06.663188 30600 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43917/jwks.json
I20260925 11:01:06.672175 30600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:38587
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:38587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43917/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:06.673429 30600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:06.675205 30600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:06.689962 30605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:06.689981 30608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:06.689926 30606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:06.690855 30600 server_base.cc:1034] running on GCE node
I20260925 11:01:06.691618 30600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:06.693001 30600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:06.694219 30600 hybrid_clock.cc:648] HybridClock initialized: now 1790334066694145 us; error 33 us; skew 500 ppm
I20260925 11:01:06.694554 30600 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:06.700188 30600 webserver.cc:460] Webserver started at http://127.24.159.254:40295/ using document root <none> and password file <none>
I20260925 11:01:06.700875 30600 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:06.700968 30600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:06.701261 30600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:06.704073 30600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "a7170c9504874a67b9272a15d5200ffc"
format_stamp: "Formatted at 2026-09-25 11:01:06 on dist-test-slave-trms"
server_key: "7ea68e225e5fc9998c28148972953b44"
server_key_iv: "c02c3e99b69bccf3eff76c5b0df27721"
server_key_version: "encryptionkey@0"
I20260925 11:01:06.704823 30600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "a7170c9504874a67b9272a15d5200ffc"
format_stamp: "Formatted at 2026-09-25 11:01:06 on dist-test-slave-trms"
server_key: "7ea68e225e5fc9998c28148972953b44"
server_key_iv: "c02c3e99b69bccf3eff76c5b0df27721"
server_key_version: "encryptionkey@0"
I20260925 11:01:06.709638 30600 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260925 11:01:06.712958 30614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:06.714313 30600 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:06.714473 30600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "a7170c9504874a67b9272a15d5200ffc"
format_stamp: "Formatted at 2026-09-25 11:01:06 on dist-test-slave-trms"
server_key: "7ea68e225e5fc9998c28148972953b44"
server_key_iv: "c02c3e99b69bccf3eff76c5b0df27721"
server_key_version: "encryptionkey@0"
I20260925 11:01:06.714658 30600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:06.761325 30600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:06.762180 30600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:06.762486 30600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:06.778694 30600 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:38587
I20260925 11:01:06.778716 30665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:38587 every 8 connection(s)
I20260925 11:01:06.779973 30600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:06.783041 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 30600
I20260925 11:01:06.783236 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:06.783627 30666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:06.783820 25215 external_mini_cluster.cc:1428] Setting key 548ca4087475e3b3a6023ea358bf116e
I20260925 11:01:06.795629 30666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc: Bootstrap starting.
I20260925 11:01:06.796464 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:06.785584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.24.159.254:38587)
0925 11:01:06.786613 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:06.786624 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:06.786665 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:01:06.788213 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:06.788235 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:06.788718 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:06.789080 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:06.789659 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.790975 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.792232 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.792959 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.793148 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:06.794253 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:06.794263 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:06.794281 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:06.794797 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:06.794886 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:06.794912 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:06.795008 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:06.795249 (+ 241us) server_negotiation.cc:299] Negotiation successful
0925 11:01:06.795541 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":561,"thread_start_us":338,"threads_started":1}
I20260925 11:01:06.799043 30666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:06.800139 30666 log.cc:824] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:06.800218 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:01:06.802832 30666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc: No bootstrap required, opened a new log
I20260925 11:01:06.809827 30666 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7170c9504874a67b9272a15d5200ffc" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38587 } }
I20260925 11:01:06.810173 30666 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:06.810286 30666 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7170c9504874a67b9272a15d5200ffc, State: Initialized, Role: FOLLOWER
I20260925 11:01:06.810828 30666 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7170c9504874a67b9272a15d5200ffc" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38587 } }
I20260925 11:01:06.811058 30666 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:06.811192 30666 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:06.811379 30666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:06.813467 30666 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7170c9504874a67b9272a15d5200ffc" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38587 } }
I20260925 11:01:06.813937 30666 leader_election.cc:302] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7170c9504874a67b9272a15d5200ffc; no voters:
I20260925 11:01:06.814520 30666 leader_election.cc:288] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:06.814636 30671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:06.815285 30671 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [term 1 LEADER]: Becoming Leader. State: Replica: a7170c9504874a67b9272a15d5200ffc, State: Running, Role: LEADER
I20260925 11:01:06.815769 30671 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7170c9504874a67b9272a15d5200ffc" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38587 } }
I20260925 11:01:06.816433 30666 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:06.818252 30672 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7170c9504874a67b9272a15d5200ffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7170c9504874a67b9272a15d5200ffc" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38587 } } }
I20260925 11:01:06.818558 30672 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:06.818364 30673 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7170c9504874a67b9272a15d5200ffc. Latest consensus state: current_term: 1 leader_uuid: "a7170c9504874a67b9272a15d5200ffc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7170c9504874a67b9272a15d5200ffc" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 38587 } } }
I20260925 11:01:06.818682 30673 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:06.820338 30676 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:06.824832 30676 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:06.833215 30676 catalog_manager.cc:1347] Generated new cluster ID: ad4e99bc796d4211a5230f9101c9baba
I20260925 11:01:06.833330 30676 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:06.845819 30676 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:06.847653 30676 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:06.862079 30676 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7170c9504874a67b9272a15d5200ffc: Generated new TSK 0
I20260925 11:01:06.863175 30676 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:01:06.892077 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:06.883810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51488 (local address 127.24.159.254:38587)
0925 11:01:06.884003 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:06.884010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:06.884393 (+ 383us) server_negotiation.cc:407] Connection header received
0925 11:01:06.884578 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:06.884584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:06.884682 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:06.884854 (+ 172us) server_negotiation.cc:226] Negotiated authn=JWT
0925 11:01:06.885266 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.886438 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.887775 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.888455 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.888580 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:06.889054 (+ 474us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 11:01:06.891447 (+ 2393us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43917/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 11:01:06.891774 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43917/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":62}
W20260925 11:01:06.892205 30669 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51488: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43917/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260925 11:01:06.893688 30695 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:06.883674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:38587 (local address 127.0.0.1:51488)
0925 11:01:06.884283 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:06.884303 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:06.884419 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:06.884850 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:06.884853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:06.884862 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0925 11:01:06.885131 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:06.885139 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.886561 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.886565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:06.887641 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:06.887652 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:06.888617 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:06.888621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:06.888756 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:06.888789 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 11:01:06.891662 (+ 2873us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43917/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0925 11:01:06.893284 (+ 1622us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.159.254:38587: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43917/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":1331,"thread_start_us":351,"threads_started":1}
I20260925 11:01:06.894682 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 30600
I20260925 11:01:06.895900 30661 generic_service.cc:196] Checking for leaks (request via RPC)
E20260925 11:01:06.903285 30696 webserver.cc:533] Webserver: sslize failed:
E20260925 11:01:06.903386 30696 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260925 11:01:06.985342 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 30600
2026-09-25T11:01:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (712 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-25T11:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:07Z Disabled control of system clock
I20260925 11:01:07.059255 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:46815
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:46145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:46815
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/testchainca.cert with env {}
W20260925 11:01:07.366080 30701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:07.366385 30701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:07.366510 30701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:07.373697 30701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:07.373802 30701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:07.373863 30701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:07.373943 30701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:07.374001 30701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260925 11:01:07.374084 30701 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260925 11:01:07.374128 30701 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260925 11:01:07.384099 30701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:46145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:46815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:46815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:07.385394 30701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:07.386682 30701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:07.397807 30706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:07.397755 30707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:07.397836 30709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:07.398624 30701 server_base.cc:1034] running on GCE node
I20260925 11:01:07.399338 30701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:07.400731 30701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:07.401935 30701 hybrid_clock.cc:648] HybridClock initialized: now 1790334067401892 us; error 34 us; skew 500 ppm
I20260925 11:01:07.402293 30701 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:07.404469 30701 webserver.cc:460] Webserver started at http://127.24.159.254:36051/ using document root <none> and password file <none>
I20260925 11:01:07.405117 30701 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:07.405211 30701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:07.405505 30701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:07.408353 30701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "8749389628f243a8a0ca1c6d31fbb7c5"
format_stamp: "Formatted at 2026-09-25 11:01:07 on dist-test-slave-trms"
server_key: "834c36115fd3b759b032a6683768584a"
server_key_iv: "f517d3398985b19dfb103ad0b10431a9"
server_key_version: "encryptionkey@0"
I20260925 11:01:07.409147 30701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "8749389628f243a8a0ca1c6d31fbb7c5"
format_stamp: "Formatted at 2026-09-25 11:01:07 on dist-test-slave-trms"
server_key: "834c36115fd3b759b032a6683768584a"
server_key_iv: "f517d3398985b19dfb103ad0b10431a9"
server_key_version: "encryptionkey@0"
I20260925 11:01:07.413961 30701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260925 11:01:07.417232 30715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:07.418614 30701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:07.418782 30701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "8749389628f243a8a0ca1c6d31fbb7c5"
format_stamp: "Formatted at 2026-09-25 11:01:07 on dist-test-slave-trms"
server_key: "834c36115fd3b759b032a6683768584a"
server_key_iv: "f517d3398985b19dfb103ad0b10431a9"
server_key_version: "encryptionkey@0"
I20260925 11:01:07.418973 30701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:07.433830 30701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:07.434731 30701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:07.434980 30701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:07.450907 30701 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:46815
I20260925 11:01:07.450930 30766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:46815 every 8 connection(s)
I20260925 11:01:07.452222 30701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:07.455101 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 30701
I20260925 11:01:07.455253 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:07.455834 25215 external_mini_cluster.cc:1428] Setting key a9661c3b75f99d739a188c421d427260
I20260925 11:01:07.455878 30767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:07.467918 30767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5: Bootstrap starting.
I20260925 11:01:07.468467 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:07.457553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.24.159.254:46815)
0925 11:01:07.458515 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:07.458526 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:07.458568 (+ 42us) server_negotiation.cc:407] Connection header received
0925 11:01:07.460346 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:07.460366 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:07.460815 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:07.461194 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:07.461749 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:07.463128 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:07.464324 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:07.465003 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:07.465177 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:07.466305 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:07.466313 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:07.466326 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:07.466827 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:07.466901 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:07.466921 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:07.467031 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:07.467236 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 11:01:07.467522 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":303,"threads_started":1}
I20260925 11:01:07.471289 30767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:07.472062 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:01:07.472329 30767 log.cc:824] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:07.474988 30767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5: No bootstrap required, opened a new log
I20260925 11:01:07.482106 30767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8749389628f243a8a0ca1c6d31fbb7c5" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46815 } }
I20260925 11:01:07.482481 30767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:07.482599 30767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8749389628f243a8a0ca1c6d31fbb7c5, State: Initialized, Role: FOLLOWER
I20260925 11:01:07.483137 30767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8749389628f243a8a0ca1c6d31fbb7c5" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46815 } }
I20260925 11:01:07.483337 30767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:07.483460 30767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:07.483609 30767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:07.485747 30767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8749389628f243a8a0ca1c6d31fbb7c5" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46815 } }
I20260925 11:01:07.486249 30767 leader_election.cc:302] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8749389628f243a8a0ca1c6d31fbb7c5; no voters:
I20260925 11:01:07.486802 30767 leader_election.cc:288] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:07.486939 30772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:07.487569 30772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [term 1 LEADER]: Becoming Leader. State: Replica: 8749389628f243a8a0ca1c6d31fbb7c5, State: Running, Role: LEADER
I20260925 11:01:07.488054 30772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8749389628f243a8a0ca1c6d31fbb7c5" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46815 } }
I20260925 11:01:07.488744 30767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:07.490621 30773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8749389628f243a8a0ca1c6d31fbb7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8749389628f243a8a0ca1c6d31fbb7c5" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46815 } } }
I20260925 11:01:07.491091 30774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8749389628f243a8a0ca1c6d31fbb7c5. Latest consensus state: current_term: 1 leader_uuid: "8749389628f243a8a0ca1c6d31fbb7c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8749389628f243a8a0ca1c6d31fbb7c5" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46815 } } }
I20260925 11:01:07.491451 30774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:07.491461 30773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:07.493489 30780 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:07.498394 30780 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:07.506618 30780 catalog_manager.cc:1347] Generated new cluster ID: fd19963ba5464f26ab37a2b53ed775fa
I20260925 11:01:07.506716 30780 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:07.516846 30780 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:07.518616 30780 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:07.526335 30780 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8749389628f243a8a0ca1c6d31fbb7c5: Generated new TSK 0
I20260925 11:01:07.527411 30780 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:01:07.662026 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:07.545048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.24.159.254:46815)
0925 11:01:07.545237 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:07.545244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:07.545696 (+ 452us) server_negotiation.cc:407] Connection header received
0925 11:01:07.545895 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:07.545901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:07.546004 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:07.546166 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0925 11:01:07.546593 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:07.547701 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:07.549152 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:07.549843 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:07.549983 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:07.550387 (+ 404us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0925 11:01:07.661275 (+110888us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 11:01:07.661718 (+ 443us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42444: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":55}
W20260925 11:01:07.662143 30796 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:07.545010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:46815 (local address 127.0.0.1:42444)
0925 11:01:07.545599 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:07.545615 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:07.545730 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:07.546171 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:07.546174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:07.546183 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0925 11:01:07.546453 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:07.546462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:07.547822 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:07.547825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:07.549007 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:07.549021 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:07.549985 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:07.549990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:07.550103 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:07.550134 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0925 11:01:07.661588 (+111454us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0925 11:01:07.661832 (+ 244us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.159.254:46815: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":315,"threads_started":1}
W20260925 11:01:07.662168 30770 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42444: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260925 11:01:07.663347 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 30701
I20260925 11:01:07.664464 30762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:07.764361 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 30701
2026-09-25T11:01:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (758 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (392 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1359 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:01:09 dist-test-slave-trms krb5kdc[30812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:01:09 dist-test-slave-trms krb5kdc[30812](info): set up 2 sockets
Sep 25 11:01:09 dist-test-slave-trms krb5kdc[30812](info): commencing operation
krb5kdc: starting...
W20260925 11:01:10.498805 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.950s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:01:10 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:01:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:01:10.681341 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43125
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:43125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:10.990305 30825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:10.990592 30825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:10.990720 30825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:10.998090 30825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:10.998219 30825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:10.998282 30825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:10.998363 30825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:10.998421 30825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:11.008554 30825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:43125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:11.009860 30825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:11.011178 30825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:11.022059 30833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:11.023056 30825 server_base.cc:1034] running on GCE node
W20260925 11:01:11.023191 30831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:11.023234 30830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:11.024058 30825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:11.025377 30825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:11.026563 30825 hybrid_clock.cc:648] HybridClock initialized: now 1790334071026518 us; error 30 us; skew 500 ppm
I20260925 11:01:11.026863 30825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:11 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334071, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:11.029861 30825 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:01:11.031235 30825 webserver.cc:460] Webserver started at http://127.24.159.254:34443/ using document root <none> and password file <none>
I20260925 11:01:11.031858 30825 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:11.031963 30825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:11.032208 30825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:11.035061 30825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "5a74041f5b3c4f2ebbf24cc835465a5f"
format_stamp: "Formatted at 2026-09-25 11:01:11 on dist-test-slave-trms"
server_key: "8d71ed121c04da7ad83d3b98351f3708"
server_key_iv: "831d7b494aff2c07f84dd25ab6fda9e5"
server_key_version: "encryptionkey@0"
I20260925 11:01:11.035758 30825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "5a74041f5b3c4f2ebbf24cc835465a5f"
format_stamp: "Formatted at 2026-09-25 11:01:11 on dist-test-slave-trms"
server_key: "8d71ed121c04da7ad83d3b98351f3708"
server_key_iv: "831d7b494aff2c07f84dd25ab6fda9e5"
server_key_version: "encryptionkey@0"
I20260925 11:01:11.040650 30825 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:01:11.043918 30840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:11.045205 30825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:01:11.045344 30825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "5a74041f5b3c4f2ebbf24cc835465a5f"
format_stamp: "Formatted at 2026-09-25 11:01:11 on dist-test-slave-trms"
server_key: "8d71ed121c04da7ad83d3b98351f3708"
server_key_iv: "831d7b494aff2c07f84dd25ab6fda9e5"
server_key_version: "encryptionkey@0"
I20260925 11:01:11.045500 30825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:11.078751 30825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:11.082942 30825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:11.083173 30825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:11.099740 30825 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:43125
I20260925 11:01:11.099763 30891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:43125 every 8 connection(s)
I20260925 11:01:11.101063 30825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:11.104696 30892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:11.107821 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 30825
I20260925 11:01:11.108003 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:11.108577 25215 external_mini_cluster.cc:1428] Setting key a75bc738362ef050f21711b21f351d22
I20260925 11:01:11.116016 30892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f: Bootstrap starting.
I20260925 11:01:11.118907 30892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:11.119846 30892 log.cc:824] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f: Log is configured to *not* fsync() on all Append() calls
Sep 25 11:01:11 dist-test-slave-trms krb5kdc[30812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:11.122704 30892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f: No bootstrap required, opened a new log
I20260925 11:01:11.130126 30895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:11.110464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55622 (local address 127.24.159.254:43125)
0925 11:01:11.111179 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:11.111191 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:11.111230 (+ 39us) server_negotiation.cc:407] Connection header received
0925 11:01:11.112822 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:11.112844 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:11.113252 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:11.113613 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:11.115015 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:11.116316 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:11.117519 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:11.118167 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:11.118358 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:11.122093 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:11.122104 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:11.122122 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:11.122173 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:11.125819 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:11.126485 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:11.126497 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:11.126506 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:11.126708 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:11.127027 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:11.127031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:11.127035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:11.127707 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:11.128017 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:11.128303 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 11:01:11.128750 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":327,"threads_started":1}
I20260925 11:01:11.132791 30892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a74041f5b3c4f2ebbf24cc835465a5f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43125 } }
I20260925 11:01:11.133137 30892 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:11.133232 30892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a74041f5b3c4f2ebbf24cc835465a5f, State: Initialized, Role: FOLLOWER
I20260925 11:01:11.133754 30892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a74041f5b3c4f2ebbf24cc835465a5f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43125 } }
I20260925 11:01:11.133966 30892 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:11.134091 30892 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:11.134262 30892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:11.136307 30892 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a74041f5b3c4f2ebbf24cc835465a5f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43125 } }
I20260925 11:01:11.136777 30892 leader_election.cc:302] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a74041f5b3c4f2ebbf24cc835465a5f; no voters:
I20260925 11:01:11.137337 30892 leader_election.cc:288] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:11.137457 30897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:11.138058 30897 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [term 1 LEADER]: Becoming Leader. State: Replica: 5a74041f5b3c4f2ebbf24cc835465a5f, State: Running, Role: LEADER
I20260925 11:01:11.138561 30897 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a74041f5b3c4f2ebbf24cc835465a5f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43125 } }
I20260925 11:01:11.139467 30892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:11.141274 30899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a74041f5b3c4f2ebbf24cc835465a5f. Latest consensus state: current_term: 1 leader_uuid: "5a74041f5b3c4f2ebbf24cc835465a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a74041f5b3c4f2ebbf24cc835465a5f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43125 } } }
I20260925 11:01:11.141709 30899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:11.141573 30898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a74041f5b3c4f2ebbf24cc835465a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a74041f5b3c4f2ebbf24cc835465a5f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43125 } } }
I20260925 11:01:11.141835 30898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:11.143342 30905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:11.147907 30905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:11.155846 30905 catalog_manager.cc:1347] Generated new cluster ID: 0d4ba18586a6418b8dcb0deff1df5063
I20260925 11:01:11.155968 30905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:11.169584 30905 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:11.171437 30905 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:11.187389 30905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a74041f5b3c4f2ebbf24cc835465a5f: Generated new TSK 0
I20260925 11:01:11.188516 30905 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:01:11.233304 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43125
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:11.552919 30918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:11.553198 30918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:11.553296 30918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:11.560485 30918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:11.560596 30918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:11.560717 30918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:11.570293 30918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:11.571501 30918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:11.572744 30918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:11.585191 30923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:11.585119 30926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:11.585160 30924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:11.585999 30918 server_base.cc:1034] running on GCE node
I20260925 11:01:11.586659 30918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:11.587564 30918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:11.588765 30918 hybrid_clock.cc:648] HybridClock initialized: now 1790334071588709 us; error 53 us; skew 500 ppm
I20260925 11:01:11.589068 30918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:11 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334071, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:11.592047 30918 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:01:11.593449 30918 webserver.cc:460] Webserver started at http://127.24.159.193:33895/ using document root <none> and password file <none>
I20260925 11:01:11.594110 30918 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:11.594249 30918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:11.594539 30918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:11.597335 30918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "c631f77f28424b269c4e439633785ade"
format_stamp: "Formatted at 2026-09-25 11:01:11 on dist-test-slave-trms"
server_key: "9204a21874f290fb7df8f3dfe5d72066"
server_key_iv: "7dd262429fbec215fb8007ae05554395"
server_key_version: "encryptionkey@0"
I20260925 11:01:11.598074 30918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "c631f77f28424b269c4e439633785ade"
format_stamp: "Formatted at 2026-09-25 11:01:11 on dist-test-slave-trms"
server_key: "9204a21874f290fb7df8f3dfe5d72066"
server_key_iv: "7dd262429fbec215fb8007ae05554395"
server_key_version: "encryptionkey@0"
I20260925 11:01:11.603021 30918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:01:11.606292 30933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:11.607539 30918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:01:11.607686 30918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "c631f77f28424b269c4e439633785ade"
format_stamp: "Formatted at 2026-09-25 11:01:11 on dist-test-slave-trms"
server_key: "9204a21874f290fb7df8f3dfe5d72066"
server_key_iv: "7dd262429fbec215fb8007ae05554395"
server_key_version: "encryptionkey@0"
I20260925 11:01:11.607863 30918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:11.625308 30918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:11.629232 30918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:11.629547 30918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:11.630462 30918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:11.631723 30918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:11.631793 30918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:11.631920 30918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:11.631973 30918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:11.661105 30918 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:32777
I20260925 11:01:11.661108 31045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:32777 every 8 connection(s)
I20260925 11:01:11.662369 30918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:11.671532 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 30918
I20260925 11:01:11.671701 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:01:11.672467 25215 external_mini_cluster.cc:1428] Setting key b82e88325ed8bad157d2d9f5cffd0a4c
Sep 25 11:01:11 dist-test-slave-trms krb5kdc[30812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334071, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:11.681494 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:11.664677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:48175 (local address 127.24.159.254:43125)
0925 11:01:11.665068 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:11.665072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:11.665996 (+ 924us) server_negotiation.cc:407] Connection header received
0925 11:01:11.667772 (+ 1776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:11.667777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:11.667857 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:11.667951 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:11.669472 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:11.670179 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:11.671723 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:11.672356 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:11.672492 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:11.676227 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:11.676232 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:11.676235 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:11.676268 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:11.678539 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:11.679320 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:11.679324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:11.679326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:11.679438 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:11.679837 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:11.679840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:11.679842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:11.680091 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:11.680193 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:11.680920 (+ 727us) server_negotiation.cc:299] Negotiation successful
0925 11:01:11.681137 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":205,"threads_started":1}
I20260925 11:01:11.682312 31049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:11.665158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43125 (local address 127.24.159.193:48175)
0925 11:01:11.665839 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:11.665886 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:11.667510 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:11.668197 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:11.668204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:11.668727 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:11.669303 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:11.669320 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:11.670348 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:11.670351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:11.671564 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:11.671575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:11.672586 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:11.672594 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:11.672792 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:11.673396 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:11.673423 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:11.676005 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:11.678664 (+ 2659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:11.678674 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:11.678691 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:11.679212 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:11.679526 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:11.679531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:11.679534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:11.679750 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:11.680197 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:11.680204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:11.680577 (+ 373us) client_negotiation.cc:769] Sending connection context
0925 11:01:11.680803 (+ 226us) client_negotiation.cc:240] Negotiation successful
0925 11:01:11.681183 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":270,"threads_started":1}
I20260925 11:01:11.683725 31046 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43125
I20260925 11:01:11.684000 31046 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:11.684689 31046 heartbeater.cc:511] Master 127.24.159.254:43125 requested a full tablet report, sending...
I20260925 11:01:11.686718 30857 ts_manager.cc:194] Registered new tserver with Master: c631f77f28424b269c4e439633785ade (127.24.159.193:32777)
I20260925 11:01:11.688301 30857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:48175
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:01:11.714645 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43125
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:12.020848 31052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:12.021122 31052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:12.021237 31052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:12.029282 31052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:12.029417 31052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:12.029603 31052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:12.039741 31052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:12.040977 31052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:12.042259 31052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:12.054865 31060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:12.055001 31057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:12.055015 31058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:12.055976 31052 server_base.cc:1034] running on GCE node
I20260925 11:01:12.056537 31052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:12.057396 31052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:12.058616 31052 hybrid_clock.cc:648] HybridClock initialized: now 1790334072058561 us; error 40 us; skew 500 ppm
I20260925 11:01:12.058919 31052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:12.066664 31052 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:01:12.068125 31052 webserver.cc:460] Webserver started at http://127.24.159.194:45867/ using document root <none> and password file <none>
I20260925 11:01:12.068807 31052 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:12.068924 31052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:12.069216 31052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:12.072180 31052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "171ea5f23bcc4a40bbb98d81c8af4426"
format_stamp: "Formatted at 2026-09-25 11:01:12 on dist-test-slave-trms"
server_key: "92a8993564e9f0c1e615d6daecfa9b2d"
server_key_iv: "26ec8bd4b3494ad23eb7b676e485f2c4"
server_key_version: "encryptionkey@0"
I20260925 11:01:12.072932 31052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "171ea5f23bcc4a40bbb98d81c8af4426"
format_stamp: "Formatted at 2026-09-25 11:01:12 on dist-test-slave-trms"
server_key: "92a8993564e9f0c1e615d6daecfa9b2d"
server_key_iv: "26ec8bd4b3494ad23eb7b676e485f2c4"
server_key_version: "encryptionkey@0"
I20260925 11:01:12.077958 31052 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 11:01:12.081331 31067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:12.082719 31052 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:12.082875 31052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "171ea5f23bcc4a40bbb98d81c8af4426"
format_stamp: "Formatted at 2026-09-25 11:01:12 on dist-test-slave-trms"
server_key: "92a8993564e9f0c1e615d6daecfa9b2d"
server_key_iv: "26ec8bd4b3494ad23eb7b676e485f2c4"
server_key_version: "encryptionkey@0"
I20260925 11:01:12.083062 31052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:12.098706 31052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:12.103739 31052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:12.104040 31052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:12.105154 31052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:12.106770 31052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:12.106863 31052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:12.106981 31052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:12.107045 31052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:12.139122 31052 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:33391
I20260925 11:01:12.139153 31179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:33391 every 8 connection(s)
I20260925 11:01:12.140395 31052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:12.141399 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 31052
I20260925 11:01:12.141569 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:01:12.142010 25215 external_mini_cluster.cc:1428] Setting key b882b31f4ec3daebcc3ffcf0c6d0b107
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:12.161120 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.143015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:47779 (local address 127.24.159.254:43125)
0925 11:01:12.143171 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.143176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.144524 (+ 1348us) server_negotiation.cc:407] Connection header received
0925 11:01:12.146369 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.146375 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.146457 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.146539 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.148292 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.149284 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.151126 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.151532 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.151625 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.155574 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:12.155579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:12.155583 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:12.155623 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:12.158069 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.158969 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.158972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.158975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.159096 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.159543 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.159546 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.159548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.159782 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:12.159893 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:12.160729 (+ 836us) server_negotiation.cc:299] Negotiation successful
0925 11:01:12.160894 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
I20260925 11:01:12.162542 31182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.143557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43125 (local address 127.24.159.194:47779)
0925 11:01:12.144385 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:12.144423 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:12.146074 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:12.146795 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:12.146804 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:12.147386 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:12.148097 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:12.148119 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.149442 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.149447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:12.150992 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:12.151007 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.151751 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.151758 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:12.152049 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.152713 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:12.152741 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:12.155353 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:12.158237 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:12.158247 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:12.158265 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:12.158862 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:12.159193 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:12.159198 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:12.159201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:12.159447 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:12.159895 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:12.159903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:12.160360 (+ 457us) client_negotiation.cc:769] Sending connection context
0925 11:01:12.160679 (+ 319us) client_negotiation.cc:240] Negotiation successful
0925 11:01:12.161240 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":341,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
I20260925 11:01:12.164115 31180 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43125
I20260925 11:01:12.164415 31180 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:12.165127 31180 heartbeater.cc:511] Master 127.24.159.254:43125 requested a full tablet report, sending...
I20260925 11:01:12.166585 30857 ts_manager.cc:194] Registered new tserver with Master: 171ea5f23bcc4a40bbb98d81c8af4426 (127.24.159.194:33391)
I20260925 11:01:12.167382 30857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:47779
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:01:12.188604 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43125
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:12.498512 31185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:12.498807 31185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:12.498930 31185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:12.506017 31185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:12.506145 31185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:12.506312 31185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:12.516003 31185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43125
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:12.517285 31185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:12.518602 31185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:12.531051 31190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:12.531085 31193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:12.531060 31191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:12.531921 31185 server_base.cc:1034] running on GCE node
I20260925 11:01:12.532528 31185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:12.533486 31185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:12.534679 31185 hybrid_clock.cc:648] HybridClock initialized: now 1790334072534621 us; error 54 us; skew 500 ppm
I20260925 11:01:12.534988 31185 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:12.538026 31185 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:01:12.539402 31185 webserver.cc:460] Webserver started at http://127.24.159.195:42865/ using document root <none> and password file <none>
I20260925 11:01:12.540076 31185 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:12.540172 31185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:12.540455 31185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:12.543370 31185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "b59a07723a42406eafd403d328d0495f"
format_stamp: "Formatted at 2026-09-25 11:01:12 on dist-test-slave-trms"
server_key: "c7a1abb9bb648f68e63bfbf60d53835d"
server_key_iv: "c00f8544f8e53f275deb7d0405cc05e8"
server_key_version: "encryptionkey@0"
I20260925 11:01:12.544204 31185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "b59a07723a42406eafd403d328d0495f"
format_stamp: "Formatted at 2026-09-25 11:01:12 on dist-test-slave-trms"
server_key: "c7a1abb9bb648f68e63bfbf60d53835d"
server_key_iv: "c00f8544f8e53f275deb7d0405cc05e8"
server_key_version: "encryptionkey@0"
I20260925 11:01:12.549624 31185 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 11:01:12.553279 31200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:12.554669 31185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:12.554812 31185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "b59a07723a42406eafd403d328d0495f"
format_stamp: "Formatted at 2026-09-25 11:01:12 on dist-test-slave-trms"
server_key: "c7a1abb9bb648f68e63bfbf60d53835d"
server_key_iv: "c00f8544f8e53f275deb7d0405cc05e8"
server_key_version: "encryptionkey@0"
I20260925 11:01:12.555009 31185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:12.565837 31185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:12.569569 31185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:12.569824 31185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:12.570803 31185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:12.572115 31185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:12.572185 31185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:12.572290 31185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:12.572342 31185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:12.601562 31185 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:38447
I20260925 11:01:12.601603 31312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:38447 every 8 connection(s)
I20260925 11:01:12.602836 31185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:12.605033 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 31185
I20260925 11:01:12.605180 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:01:12.605716 25215 external_mini_cluster.cc:1428] Setting key ed8b8193914ea542cc11d1dc2779a977
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:12.621800 31048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.605298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:39351 (local address 127.24.159.254:43125)
0925 11:01:12.605557 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.605571 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.606844 (+ 1273us) server_negotiation.cc:407] Connection header received
0925 11:01:12.608832 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.608838 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.608922 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.609012 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.610568 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.611377 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.612917 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.613331 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.613443 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.616601 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:12.616606 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:12.616610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:12.616644 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:12.619006 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.619739 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.619743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.619746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.619862 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.620302 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.620305 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.620307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.620642 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:12.620769 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:12.621393 (+ 624us) server_negotiation.cc:299] Negotiation successful
0925 11:01:12.621570 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260925 11:01:12.622750 31315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.605859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43125 (local address 127.24.159.195:39351)
0925 11:01:12.606684 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:12.606739 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:12.608553 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:12.609264 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:12.609272 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:12.609788 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:12.610405 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:12.610425 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.611531 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.611535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:12.612779 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:12.612791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.613549 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.613557 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:12.613775 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.614371 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:12.614392 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:12.616405 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:12.619132 (+ 2727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:12.619141 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:12.619158 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:12.619593 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:12.619967 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:12.619972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:12.619976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:12.620197 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:12.620770 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:12.620778 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:12.621089 (+ 311us) client_negotiation.cc:769] Sending connection context
0925 11:01:12.621310 (+ 221us) client_negotiation.cc:240] Negotiation successful
0925 11:01:12.621685 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":324,"threads_started":1}
I20260925 11:01:12.623912 31313 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43125
I20260925 11:01:12.624194 31313 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:12.624841 31313 heartbeater.cc:511] Master 127.24.159.254:43125 requested a full tablet report, sending...
I20260925 11:01:12.626108 30857 ts_manager.cc:194] Registered new tserver with Master: b59a07723a42406eafd403d328d0495f (127.24.159.195:38447)
I20260925 11:01:12.626895 30857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:39351
I20260925 11:01:12.632702 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/test-admin.keytab.
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:12.657169 25215 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260925 11:01:12.662251 31320 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:12.659745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42548 (local address 127.24.159.193:32777)
0925 11:01:12.660298 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.660307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.660339 (+ 32us) server_negotiation.cc:407] Connection header received
0925 11:01:12.660492 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.660500 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.660759 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.660903 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.661793 (+ 890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42548: BlockingRecv error: recv got EOF from 127.0.0.1:42548 (error 108)
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":273,"threads_started":1}
W20260925 11:01:12.662305 25215 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5cc' of type FILE
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:12.664125 25215 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:12.665743 25215 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:01:12.679720 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.667593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42556 (local address 127.24.159.193:32777)
0925 11:01:12.667770 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.667776 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.667958 (+ 182us) server_negotiation.cc:407] Connection header received
0925 11:01:12.668091 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.668094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.668150 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.668218 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.669148 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.670095 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.671328 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.671856 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.671959 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.675116 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:12.675126 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:12.675140 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:12.675174 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:12.677487 (+ 2313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.678021 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.678027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.678029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.678139 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.678469 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.678472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.678474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.678794 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:12.679004 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:12.679238 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 11:01:12.679472 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260925 11:01:12.684937 31320 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:12.683412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.24.159.193:32777)
0925 11:01:12.683610 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.683616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.683732 (+ 116us) server_negotiation.cc:407] Connection header received
0925 11:01:12.683873 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.683878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.683955 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.684031 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.684727 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42562: BlockingRecv error: recv got EOF from 127.0.0.1:42562 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260925 11:01:12.685420 25215 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5cc)
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:12.686831 25215 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:12.688036 25215 init.cc:295] Successfully reacquired a new kerberos TGT
I20260925 11:01:12.691288 31046 heartbeater.cc:503] Master 127.24.159.254:43125 was elected leader, sending a full tablet report...
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:01:12.700845 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.689193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42570 (local address 127.24.159.193:32777)
0925 11:01:12.689363 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.689369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.689606 (+ 237us) server_negotiation.cc:407] Connection header received
0925 11:01:12.689715 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.689718 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.689773 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.689848 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.690684 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.691574 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.692818 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.693297 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.693389 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.696542 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:12.696546 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:12.696550 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:12.696579 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:12.698853 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.699411 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.699415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.699417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.699523 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.699791 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.699794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.699795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.700044 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:12.700220 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:12.700418 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 11:01:12.700565 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260925 11:01:12.704756 31320 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:12.703154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42578 (local address 127.24.159.193:32777)
0925 11:01:12.703310 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.703315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.703556 (+ 241us) server_negotiation.cc:407] Connection header received
0925 11:01:12.703694 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.703698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.703762 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.703837 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.704568 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42578: BlockingRecv error: recv got EOF from 127.0.0.1:42578 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260925 11:01:12.704924 25215 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790334022505657-25215-0/krb5kdc/krb5cc' of type FILE
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:12.706284 25215 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:12.707427 25215 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:01:12.719916 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.708587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42582 (local address 127.24.159.193:32777)
0925 11:01:12.708777 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.708782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.709121 (+ 339us) server_negotiation.cc:407] Connection header received
0925 11:01:12.709248 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.709251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.709305 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.709376 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.710177 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.710894 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.712124 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.712514 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.712602 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.715674 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:12.715679 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:12.715682 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:12.715711 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:12.717841 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.718429 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.718433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.718435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.718570 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.718831 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.718833 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.718835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.719092 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:12.719301 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:12.719441 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 11:01:12.719594 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 11:01:12.721256 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 30918
I20260925 11:01:12.731972 31320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.721867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42588 (local address 127.24.159.193:32777)
0925 11:01:12.722045 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.722051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.722395 (+ 344us) server_negotiation.cc:407] Connection header received
0925 11:01:12.722537 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.722542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.722594 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.722668 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.723507 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.724159 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.725344 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.725754 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.725860 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.727656 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:12.727660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:12.727664 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:12.727691 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:12.729645 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.730216 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.730222 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.730225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.730359 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.730683 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.730688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.730692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.730939 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:12.731168 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:12.731467 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 11:01:12.731667 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:01:12.732535 31041 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:12.814563 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 30918
I20260925 11:01:12.827271 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 31052
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:01:12.841320 31333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.828145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.24.159.194:33391)
0925 11:01:12.828653 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.828661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.828685 (+ 24us) server_negotiation.cc:407] Connection header received
0925 11:01:12.828760 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.828767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.828978 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.829107 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.829978 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.831030 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.832173 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.832703 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.832796 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.836210 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:12.836222 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:12.836237 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:12.836270 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:12.838972 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.839595 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.839601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.839604 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.839721 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.840020 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.840023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.840025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.840370 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:12.840597 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:12.840823 (+ 226us) server_negotiation.cc:299] Negotiation successful
0925 11:01:12.841040 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":243,"threads_started":1}
I20260925 11:01:12.842761 31175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:12.923656 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 31052
I20260925 11:01:12.936580 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 31185
Sep 25 11:01:12 dist-test-slave-trms krb5kdc[30812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334072, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:01:12.952729 31335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:12.937517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34068 (local address 127.24.159.195:38447)
0925 11:01:12.938104 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:12.938110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:12.938138 (+ 28us) server_negotiation.cc:407] Connection header received
0925 11:01:12.938232 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:12.938240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:12.938473 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:12.938597 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:12.939593 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.940703 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.941845 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:12.942481 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:12.942576 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:12.946103 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:12.946114 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:12.946129 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:12.946161 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:12.949429 (+ 3268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.950350 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.950360 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.950363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.950520 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:12.950985 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:12.950991 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:12.950994 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:12.951468 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:12.951766 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:12.952078 (+ 312us) server_negotiation.cc:299] Negotiation successful
0925 11:01:12.952327 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":285,"threads_started":1}
I20260925 11:01:12.955420 31308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:13.037029 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 31185
I20260925 11:01:13.050355 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 30825
I20260925 11:01:13.051471 30887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:13.153046 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 30825
2026-09-25T11:01:13Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3641 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:01:13 dist-test-slave-trms krb5kdc[31340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:01:13 dist-test-slave-trms krb5kdc[31340](info): set up 2 sockets
Sep 25 11:01:13 dist-test-slave-trms krb5kdc[31340](info): commencing operation
krb5kdc: starting...
W20260925 11:01:14.154867 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.965s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:01:14 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:01:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.254.keytab.
Entry for principal oryx/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.254.keytab.
I20260925 11:01:14.341045 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36795
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.254.keytab
--principal=oryx/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:36795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:14.648816 31353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:14.649123 31353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:14.649243 31353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:14.656538 31353 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:14.656658 31353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:14.656736 31353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:14.656808 31353 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:14.656868 31353 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:14.666630 31353 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:36795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.254.keytab
--principal=oryx/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:14.667995 31353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:14.669337 31353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:14.680356 31358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:14.680377 31361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:14.680294 31359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:14.681138 31353 server_base.cc:1034] running on GCE node
I20260925 11:01:14.681880 31353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:14.683292 31353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:14.684491 31353 hybrid_clock.cc:648] HybridClock initialized: now 1790334074684441 us; error 37 us; skew 500 ppm
I20260925 11:01:14.684801 31353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:14 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:14.687721 31353 init.cc:377] Logged in from keytab as oryx/127.24.159.254@KRBTEST.COM (short username oryx)
I20260925 11:01:14.689101 31353 webserver.cc:460] Webserver started at http://127.24.159.254:42029/ using document root <none> and password file <none>
I20260925 11:01:14.689728 31353 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:14.689824 31353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:14.690076 31353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:14.692978 31353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "1b678ee8f48245f7aaf9d8e30f224adb"
format_stamp: "Formatted at 2026-09-25 11:01:14 on dist-test-slave-trms"
server_key: "76148bf5d3ae5ff9efd818d96d1e7b34"
server_key_iv: "8d5ff5274888842146103993df892b5c"
server_key_version: "encryptionkey@0"
I20260925 11:01:14.693691 31353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "1b678ee8f48245f7aaf9d8e30f224adb"
format_stamp: "Formatted at 2026-09-25 11:01:14 on dist-test-slave-trms"
server_key: "76148bf5d3ae5ff9efd818d96d1e7b34"
server_key_iv: "8d5ff5274888842146103993df892b5c"
server_key_version: "encryptionkey@0"
I20260925 11:01:14.698755 31353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260925 11:01:14.701990 31368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:14.703338 31353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:14.703482 31353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "1b678ee8f48245f7aaf9d8e30f224adb"
format_stamp: "Formatted at 2026-09-25 11:01:14 on dist-test-slave-trms"
server_key: "76148bf5d3ae5ff9efd818d96d1e7b34"
server_key_iv: "8d5ff5274888842146103993df892b5c"
server_key_version: "encryptionkey@0"
I20260925 11:01:14.703644 31353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:14.733103 31353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:14.737068 31353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:14.737303 31353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:14.753448 31353 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:36795
I20260925 11:01:14.753466 31419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:36795 every 8 connection(s)
I20260925 11:01:14.754813 31353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:14.757135 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 31353
I20260925 11:01:14.757333 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:14.758124 25215 external_mini_cluster.cc:1428] Setting key 5c3ea1dff98475d3c5f232f34734511e
I20260925 11:01:14.759167 31420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 11:01:14 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:14.772754 31420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: Bootstrap starting.
I20260925 11:01:14.776125 31420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:14.777334 31420 log.cc:824] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:14.779971 31423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:14.760099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55432 (local address 127.24.159.254:36795)
0925 11:01:14.760838 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:14.760852 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:14.760910 (+ 58us) server_negotiation.cc:407] Connection header received
0925 11:01:14.762650 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:14.762678 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:14.763191 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:14.763618 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:14.764870 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:14.766897 (+ 2027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:14.768056 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:14.769053 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:14.769228 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:14.773017 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:14.773025 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:14.773041 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:14.773082 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:14.775923 (+ 2841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:14.776694 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:14.776700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:14.776707 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:14.776860 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:14.777247 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:14.777250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:14.777252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:14.777857 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:14.778153 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:14.778631 (+ 478us) server_negotiation.cc:299] Negotiation successful
0925 11:01:14.779018 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":299,"threads_started":1}
I20260925 11:01:14.780194 31420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: No bootstrap required, opened a new log
I20260925 11:01:14.787716 31420 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } }
I20260925 11:01:14.788066 31420 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:14.788163 31420 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b678ee8f48245f7aaf9d8e30f224adb, State: Initialized, Role: FOLLOWER
I20260925 11:01:14.788690 31420 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } }
I20260925 11:01:14.788915 31420 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:14.789034 31420 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:14.789176 31420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:14.791260 31420 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } }
I20260925 11:01:14.791736 31420 leader_election.cc:302] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b678ee8f48245f7aaf9d8e30f224adb; no voters:
I20260925 11:01:14.792371 31420 leader_election.cc:288] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:14.792444 31425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:14.793133 31425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 1 LEADER]: Becoming Leader. State: Replica: 1b678ee8f48245f7aaf9d8e30f224adb, State: Running, Role: LEADER
I20260925 11:01:14.793589 31425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } }
I20260925 11:01:14.794319 31420 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:14.796275 31427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b678ee8f48245f7aaf9d8e30f224adb. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } }
I20260925 11:01:14.796659 31427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:14.796548 31426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } }
I20260925 11:01:14.796842 31426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:14.798455 31434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:14.803932 31434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:14.812762 31434 catalog_manager.cc:1347] Generated new cluster ID: d15286d5e2564bee8585b41724319932
I20260925 11:01:14.812866 31434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:14.839288 31434 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:14.840826 31434 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:14.857165 31434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: Generated new TSK 0
I20260925 11:01:14.857990 31434 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab.
Entry for principal oryx/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab.
I20260925 11:01:14.909976 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36795
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab
--principal=oryx/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:15.234828 31446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:15.235095 31446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:15.235194 31446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:15.242322 31446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:15.242430 31446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:15.242548 31446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:15.251950 31446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab
--principal=oryx/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:15.253194 31446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:15.254539 31446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:15.266919 31452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:15.266897 31454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:15.266979 31451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:15.267716 31446 server_base.cc:1034] running on GCE node
I20260925 11:01:15.268323 31446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:15.269234 31446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:15.270414 31446 hybrid_clock.cc:648] HybridClock initialized: now 1790334075270357 us; error 48 us; skew 500 ppm
I20260925 11:01:15.270691 31446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:15 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334075, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:15.273581 31446 init.cc:377] Logged in from keytab as oryx/127.24.159.193@KRBTEST.COM (short username oryx)
I20260925 11:01:15.274933 31446 webserver.cc:460] Webserver started at http://127.24.159.193:34895/ using document root <none> and password file <none>
I20260925 11:01:15.275569 31446 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:15.275663 31446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:15.275904 31446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:15.278896 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "6c8452f139574a7985ec962338123efd"
format_stamp: "Formatted at 2026-09-25 11:01:15 on dist-test-slave-trms"
server_key: "de2c4eab1acbb7ad38de53b7f5e2a7cc"
server_key_iv: "d8d0a29fcd2d6e1656f65f2c310b0c23"
server_key_version: "encryptionkey@0"
I20260925 11:01:15.279749 31446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "6c8452f139574a7985ec962338123efd"
format_stamp: "Formatted at 2026-09-25 11:01:15 on dist-test-slave-trms"
server_key: "de2c4eab1acbb7ad38de53b7f5e2a7cc"
server_key_iv: "d8d0a29fcd2d6e1656f65f2c310b0c23"
server_key_version: "encryptionkey@0"
I20260925 11:01:15.284641 31446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:01:15.287832 31461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:15.289114 31446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:01:15.289240 31446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "6c8452f139574a7985ec962338123efd"
format_stamp: "Formatted at 2026-09-25 11:01:15 on dist-test-slave-trms"
server_key: "de2c4eab1acbb7ad38de53b7f5e2a7cc"
server_key_iv: "d8d0a29fcd2d6e1656f65f2c310b0c23"
server_key_version: "encryptionkey@0"
I20260925 11:01:15.289398 31446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:15.310294 31446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:15.313962 31446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:15.314217 31446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:15.315090 31446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:15.316397 31446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:15.316471 31446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:15.316565 31446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:15.316606 31446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:15.345589 31446 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:43673
I20260925 11:01:15.345633 31573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:43673 every 8 connection(s)
I20260925 11:01:15.346801 31446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:15.348306 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 31446
I20260925 11:01:15.348515 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:01:15.349226 25215 external_mini_cluster.cc:1428] Setting key f406648130e19d8712f4799ddfc88de6
Sep 25 11:01:15 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334075, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:15.368805 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:15.349488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:43059 (local address 127.24.159.254:36795)
0925 11:01:15.350180 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:15.350220 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:15.351098 (+ 878us) server_negotiation.cc:407] Connection header received
0925 11:01:15.353436 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:15.353441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:15.353505 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:15.353597 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:15.355517 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:15.356348 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:15.358526 (+ 2178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:15.359117 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:15.359278 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:15.362771 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:15.362778 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:15.362782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:15.362822 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:15.365648 (+ 2826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:15.366397 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:15.366403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:15.366406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:15.366555 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:15.366946 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:15.366951 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:15.366954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:15.367317 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:15.367477 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:15.368161 (+ 684us) server_negotiation.cc:299] Negotiation successful
0925 11:01:15.368391 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":443,"threads_started":1}
I20260925 11:01:15.369537 31577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:15.350073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.193:43059)
0925 11:01:15.350822 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:15.350861 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:15.353131 (+ 2270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:15.353865 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:15.353877 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:15.354584 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:15.355342 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:15.355367 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:15.356505 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:15.356511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:15.358360 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:15.358376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:15.359357 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:15.359363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:15.359572 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:15.360221 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:15.360248 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:15.362510 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:15.365787 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:15.365794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:15.365810 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:15.366281 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:15.366650 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:15.366653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:15.366655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:15.366834 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:15.367440 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:15.367445 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:15.367776 (+ 331us) client_negotiation.cc:769] Sending connection context
0925 11:01:15.368003 (+ 227us) client_negotiation.cc:240] Negotiation successful
0925 11:01:15.368409 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":305,"threads_started":1}
WARNING: no policy specified for oryx/127.24.159.194@KRBTEST.COM; defaulting to no policy
I20260925 11:01:15.371057 31574 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36795
I20260925 11:01:15.371322 31574 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:15.372072 31574 heartbeater.cc:511] Master 127.24.159.254:36795 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.194@KRBTEST.COM" created.
I20260925 11:01:15.374315 31385 ts_manager.cc:194] Registered new tserver with Master: 6c8452f139574a7985ec962338123efd (127.24.159.193:43673)
I20260925 11:01:15.376164 31385 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:43059
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab.
Entry for principal oryx/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab.
I20260925 11:01:15.397394 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36795
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab
--principal=oryx/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:15.705955 31580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:15.706266 31580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:15.706398 31580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:15.713645 31580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:15.713764 31580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:15.713917 31580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:15.723486 31580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab
--principal=oryx/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:15.724735 31580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:15.726003 31580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:15.738488 31585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:15.738467 31586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:15.738413 31588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:15.739082 31580 server_base.cc:1034] running on GCE node
I20260925 11:01:15.739624 31580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:15.740463 31580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:15.741658 31580 hybrid_clock.cc:648] HybridClock initialized: now 1790334075741618 us; error 30 us; skew 500 ppm
I20260925 11:01:15.741938 31580 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:15 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334075, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:15.744807 31580 init.cc:377] Logged in from keytab as oryx/127.24.159.194@KRBTEST.COM (short username oryx)
I20260925 11:01:15.746099 31580 webserver.cc:460] Webserver started at http://127.24.159.194:38205/ using document root <none> and password file <none>
I20260925 11:01:15.746755 31580 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:15.746853 31580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:15.747085 31580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:15.749827 31580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "1275fb03f9e748f281bc34fcab9cc133"
format_stamp: "Formatted at 2026-09-25 11:01:15 on dist-test-slave-trms"
server_key: "d399e64744cde32d3843145f82b5a05f"
server_key_iv: "1b16f2a1008e80ba62f6813f0ef61d28"
server_key_version: "encryptionkey@0"
I20260925 11:01:15.750577 31580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "1275fb03f9e748f281bc34fcab9cc133"
format_stamp: "Formatted at 2026-09-25 11:01:15 on dist-test-slave-trms"
server_key: "d399e64744cde32d3843145f82b5a05f"
server_key_iv: "1b16f2a1008e80ba62f6813f0ef61d28"
server_key_version: "encryptionkey@0"
I20260925 11:01:15.756093 31580 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:01:15.760203 31595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:15.761760 31580 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:01:15.761907 31580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "1275fb03f9e748f281bc34fcab9cc133"
format_stamp: "Formatted at 2026-09-25 11:01:15 on dist-test-slave-trms"
server_key: "d399e64744cde32d3843145f82b5a05f"
server_key_iv: "1b16f2a1008e80ba62f6813f0ef61d28"
server_key_version: "encryptionkey@0"
I20260925 11:01:15.762108 31580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:15.780603 31580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:15.784245 31580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:15.784485 31580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:15.785342 31580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:15.786764 31580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:15.786837 31580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:15.786929 31580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:15.786971 31580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:15.817687 31580 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:34645
I20260925 11:01:15.817724 31707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:34645 every 8 connection(s)
I20260925 11:01:15.818984 31580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:15.824120 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 31580
I20260925 11:01:15.824286 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:01:15.824690 25215 external_mini_cluster.cc:1428] Setting key f9b3cc6d6ee7c90712693e75a89f8a75
Sep 25 11:01:15 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334075, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:15.838745 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:15.821462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:57783 (local address 127.24.159.254:36795)
0925 11:01:15.821626 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:15.821630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:15.822804 (+ 1174us) server_negotiation.cc:407] Connection header received
0925 11:01:15.824621 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:15.824627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:15.824708 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:15.824791 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:15.826486 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:15.827505 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:15.829068 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:15.829618 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:15.829737 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:15.833209 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:15.833214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:15.833217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:15.833250 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:15.835670 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:15.836504 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:15.836507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:15.836509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:15.836633 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:15.837078 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:15.837080 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:15.837082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:15.837328 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:15.837445 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:15.838308 (+ 863us) server_negotiation.cc:299] Negotiation successful
0925 11:01:15.838500 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:01:15.840170 31710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:15.821917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:57783)
0925 11:01:15.822651 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:15.822696 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:15.824324 (+ 1628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:15.825051 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:15.825060 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:15.825669 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:15.826309 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:15.826338 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:15.827646 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:15.827649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:15.828928 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:15.828942 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:15.829805 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:15.829810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:15.830021 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:15.830693 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:15.830716 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:15.832980 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:15.835826 (+ 2846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:15.835835 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:15.835855 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:15.836392 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:15.836723 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:15.836728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:15.836731 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:15.836988 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:15.837444 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:15.837451 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:15.837889 (+ 438us) client_negotiation.cc:769] Sending connection context
0925 11:01:15.838230 (+ 341us) client_negotiation.cc:240] Negotiation successful
0925 11:01:15.838770 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":311,"threads_started":1}
I20260925 11:01:15.841670 31708 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36795
I20260925 11:01:15.842039 31708 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:15.843032 31708 heartbeater.cc:511] Master 127.24.159.254:36795 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.24.159.195@KRBTEST.COM; defaulting to no policy
I20260925 11:01:15.844733 31385 ts_manager.cc:194] Registered new tserver with Master: 1275fb03f9e748f281bc34fcab9cc133 (127.24.159.194:34645)
I20260925 11:01:15.845577 31385 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:57783
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab.
Entry for principal oryx/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab.
I20260925 11:01:15.869491 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36795
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab
--principal=oryx/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:16.182370 31713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:16.182649 31713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:16.182739 31713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:16.189779 31713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:16.189883 31713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:16.190006 31713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:16.199591 31713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab
--principal=oryx/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36795
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:16.200783 31713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:16.202023 31713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:16.214138 31718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:16.214119 31719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:16.214685 31721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:16.215080 31713 server_base.cc:1034] running on GCE node
I20260925 11:01:16.215670 31713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:16.216583 31713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:16.217777 31713 hybrid_clock.cc:648] HybridClock initialized: now 1790334076217724 us; error 41 us; skew 500 ppm
I20260925 11:01:16.218087 31713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:16 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:16.221014 31713 init.cc:377] Logged in from keytab as oryx/127.24.159.195@KRBTEST.COM (short username oryx)
I20260925 11:01:16.222366 31713 webserver.cc:460] Webserver started at http://127.24.159.195:35791/ using document root <none> and password file <none>
I20260925 11:01:16.223011 31713 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:16.223124 31713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:16.223400 31713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:16.226161 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "9f288844d41a440d8db0965a2ed7e6d7"
format_stamp: "Formatted at 2026-09-25 11:01:16 on dist-test-slave-trms"
server_key: "683753a8a49dd3f918fb417e5679c989"
server_key_iv: "b255358759dec45a451a0c28aa1932a4"
server_key_version: "encryptionkey@0"
I20260925 11:01:16.226953 31713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "9f288844d41a440d8db0965a2ed7e6d7"
format_stamp: "Formatted at 2026-09-25 11:01:16 on dist-test-slave-trms"
server_key: "683753a8a49dd3f918fb417e5679c989"
server_key_iv: "b255358759dec45a451a0c28aa1932a4"
server_key_version: "encryptionkey@0"
I20260925 11:01:16.231833 31713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:01:16.235169 31728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:16.236441 31713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:01:16.236586 31713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "9f288844d41a440d8db0965a2ed7e6d7"
format_stamp: "Formatted at 2026-09-25 11:01:16 on dist-test-slave-trms"
server_key: "683753a8a49dd3f918fb417e5679c989"
server_key_iv: "b255358759dec45a451a0c28aa1932a4"
server_key_version: "encryptionkey@0"
I20260925 11:01:16.236757 31713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:16.255143 31713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:16.259286 31713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:16.259563 31713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:16.260447 31713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:16.261706 31713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:16.261783 31713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:16.261904 31713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:16.261955 31713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:16.291276 31713 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:43259
I20260925 11:01:16.291325 31840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:43259 every 8 connection(s)
I20260925 11:01:16.292484 31713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:16.299242 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 31713
I20260925 11:01:16.299420 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:01:16.299851 25215 external_mini_cluster.cc:1428] Setting key 421d79828eb7f9d332d16b547c53e3a3
Sep 25 11:01:16 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:16.311151 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.294995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:57985 (local address 127.24.159.254:36795)
0925 11:01:16.295178 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:16.295183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:16.296348 (+ 1165us) server_negotiation.cc:407] Connection header received
0925 11:01:16.298216 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:16.298222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:16.298311 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:16.298405 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:16.299957 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.300736 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.302440 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.302869 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.302968 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.306033 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:16.306037 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:16.306040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:16.306074 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:16.308568 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.309251 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.309254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.309256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.309361 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.309688 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.309691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.309693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.309954 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:16.310055 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:16.310727 (+ 672us) server_negotiation.cc:299] Negotiation successful
0925 11:01:16.310908 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 11:01:16.312064 31843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.295514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.195:57985)
0925 11:01:16.296183 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:16.296222 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:16.297917 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:16.298654 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:16.298663 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:16.299188 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:16.299794 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.299813 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.300896 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.300902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.302312 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.302327 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.303066 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.303071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.303263 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.303819 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:16.303844 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:16.305837 (+ 1993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:16.308716 (+ 2879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.308723 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.308737 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.309163 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.309437 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.309440 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.309442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.309614 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.310042 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:16.310047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:16.310420 (+ 373us) client_negotiation.cc:769] Sending connection context
0925 11:01:16.310648 (+ 228us) client_negotiation.cc:240] Negotiation successful
0925 11:01:16.311034 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":249,"threads_started":1}
I20260925 11:01:16.313412 31841 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36795
I20260925 11:01:16.313697 31841 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:16.314499 31841 heartbeater.cc:511] Master 127.24.159.254:36795 requested a full tablet report, sending...
I20260925 11:01:16.315846 31385 ts_manager.cc:194] Registered new tserver with Master: 9f288844d41a440d8db0965a2ed7e6d7 (127.24.159.195:43259)
I20260925 11:01:16.316602 31385 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:57985
I20260925 11:01:16.326710 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.159.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.253.keytab.
Entry for principal oryx/127.24.159.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.253.keytab.
I20260925 11:01:16.370633 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.253:43627
--webserver_interface=127.24.159.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.253.keytab
--principal=oryx/127.24.159.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:36795,127.24.159.253:43627
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:01:16.379225 31574 heartbeater.cc:503] Master 127.24.159.254:36795 was elected leader, sending a full tablet report...
W20260925 11:01:16.686813 31846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:16.687119 31846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:16.687238 31846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:16.694459 31846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:16.694584 31846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:16.694658 31846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:16.694731 31846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:16.694787 31846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:16.704607 31846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:36795,127.24.159.253:43627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.253.keytab
--principal=oryx/127.24.159.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.253:43627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.159.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
W20260925 11:01:16.704982 31846 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.159.254:36795,127.24.159.253:43627'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260925 11:01:16.706008 31846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:16.707367 31846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:16.718918 31851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:16.718520 31852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:16.718950 31854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:16.719965 31846 server_base.cc:1034] running on GCE node
I20260925 11:01:16.720659 31846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:16.721977 31846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:16.723172 31846 hybrid_clock.cc:648] HybridClock initialized: now 1790334076723106 us; error 48 us; skew 500 ppm
I20260925 11:01:16.723479 31846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:16 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:16.726307 31846 init.cc:377] Logged in from keytab as oryx/127.24.159.253@KRBTEST.COM (short username oryx)
I20260925 11:01:16.727687 31846 webserver.cc:460] Webserver started at http://127.24.159.253:44673/ using document root <none> and password file <none>
I20260925 11:01:16.728359 31846 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:16.728456 31846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:16.728749 31846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:16.731631 31846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data/instance:
uuid: "ddf176fd1cc24080b1c99912aa360f1e"
format_stamp: "Formatted at 2026-09-25 11:01:16 on dist-test-slave-trms"
server_key: "172129c0f7b5640551d9910776112da0"
server_key_iv: "158d6a6e7a55bec3ebb0e1ee19a9cb17"
server_key_version: "encryptionkey@0"
I20260925 11:01:16.732343 31846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal/instance:
uuid: "ddf176fd1cc24080b1c99912aa360f1e"
format_stamp: "Formatted at 2026-09-25 11:01:16 on dist-test-slave-trms"
server_key: "172129c0f7b5640551d9910776112da0"
server_key_iv: "158d6a6e7a55bec3ebb0e1ee19a9cb17"
server_key_version: "encryptionkey@0"
I20260925 11:01:16.737192 31846 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260925 11:01:16.740373 31861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:16.741693 31846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:01:16.741837 31846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal
uuid: "ddf176fd1cc24080b1c99912aa360f1e"
format_stamp: "Formatted at 2026-09-25 11:01:16 on dist-test-slave-trms"
server_key: "172129c0f7b5640551d9910776112da0"
server_key_iv: "158d6a6e7a55bec3ebb0e1ee19a9cb17"
server_key_version: "encryptionkey@0"
I20260925 11:01:16.742034 31846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:16.753405 31846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:16.757074 31846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:16.757287 31846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:16.773726 31846 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.253:43627
I20260925 11:01:16.773763 31912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.253:43627 every 8 connection(s)
I20260925 11:01:16.775072 31846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
I20260925 11:01:16.778652 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 31846
I20260925 11:01:16.778736 31913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:16.787863 31913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:01:16 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.253@KRBTEST.COM
Sep 25 11:01:16 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.253@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:16.801596 31916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.780837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.24.159.253:43627)
0925 11:01:16.781725 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:16.781740 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:16.782050 (+ 310us) server_negotiation.cc:407] Connection header received
0925 11:01:16.783606 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:16.783627 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:16.784007 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:16.784327 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:16.786013 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.787279 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.788976 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.789641 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.789759 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.794134 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:16.794142 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:16.794153 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:16.794221 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:16.797246 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.797979 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.797989 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.797995 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.798155 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.798565 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.798568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.798570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.799157 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:16.799394 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:16.799861 (+ 467us) server_negotiation.cc:299] Negotiation successful
0925 11:01:16.800269 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":373,"threads_started":1}
I20260925 11:01:16.805133 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.790706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55440 (local address 127.24.159.254:36795)
0925 11:01:16.790873 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:16.790877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:16.791518 (+ 641us) server_negotiation.cc:407] Connection header received
0925 11:01:16.791682 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:16.791688 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:16.791763 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:16.791839 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:16.792952 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.793773 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.795616 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.796259 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.796396 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.799618 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:16.799632 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:16.799638 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:16.799695 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:16.802493 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.803180 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.803185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.803188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.803332 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.803635 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.803640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.803642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.804043 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:16.804172 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:16.804626 (+ 454us) server_negotiation.cc:299] Negotiation successful
0925 11:01:16.804827 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:01:16.805212 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.790716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:55440)
0925 11:01:16.791391 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:16.791422 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:16.791569 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:16.791904 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:16.791911 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:16.792515 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:16.792807 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.792820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.794043 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.794049 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.795479 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.795491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.796435 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.796441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.796564 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.797061 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:16.797077 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:16.799431 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:16.802602 (+ 3171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.802611 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.802617 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.803072 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.803424 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.803427 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.803429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.803548 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.804157 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:16.804164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:16.804342 (+ 178us) client_negotiation.cc:769] Sending connection context
0925 11:01:16.804520 (+ 178us) client_negotiation.cc:240] Negotiation successful
0925 11:01:16.804811 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":369,"threads_started":1}
I20260925 11:01:16.806670 31913 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:01:16 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334076, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.253@KRBTEST.COM for oryx/127.24.159.253@KRBTEST.COM
I20260925 11:01:16.817499 31917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.807408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.0.0.1:43146)
0925 11:01:16.807573 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:16.807591 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:16.807681 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:16.807981 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:16.807984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:16.808264 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:16.808388 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.808395 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.809281 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.809284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.810108 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.810118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.810766 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.810770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.810887 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.811348 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:16.811364 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:16.813159 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:16.815697 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.815700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.815703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.816042 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.816305 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.816308 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.816311 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.816419 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.816826 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:16.816829 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:16.816969 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 11:01:16.817067 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 11:01:16.817259 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 11:01:16.817746 31916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.807417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.24.159.253:43627)
0925 11:01:16.807550 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:16.807554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:16.807644 (+ 90us) server_negotiation.cc:407] Connection header received
0925 11:01:16.807807 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:16.807811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:16.807865 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:16.807943 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:16.808493 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.809168 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.810249 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.810638 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.810728 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.813318 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:16.813323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:16.813326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:16.813352 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:16.815599 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.816130 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.816133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.816135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.816234 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.816496 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.816499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.816501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.816740 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:16.816840 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:16.817196 (+ 356us) server_negotiation.cc:299] Negotiation successful
0925 11:01:16.817414 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 11:01:16.825030 31913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: Bootstrap starting.
I20260925 11:01:16.827955 31913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:16.828930 31913 log.cc:824] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:16.831461 31913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: No bootstrap required, opened a new log
I20260925 11:01:16.838167 31913 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } }
I20260925 11:01:16.838527 31913 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:16.838622 31913 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddf176fd1cc24080b1c99912aa360f1e, State: Initialized, Role: FOLLOWER
I20260925 11:01:16.839192 31913 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } }
I20260925 11:01:16.839990 31919 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } } }
I20260925 11:01:16.840310 31919 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:01:16.841037 31913 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:16.848379 31708 heartbeater.cc:503] Master 127.24.159.254:36795 was elected leader, sending a full tablet report...
W20260925 11:01:16.849346 31930 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:01:16.849532 31930 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:01:16.851956 31846 master_runner.cc:428] Detected that this master ddf176fd1cc24080b1c99912aa360f1e is joining an existing cluster
I20260925 11:01:16.852037 31846 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.159.253:43627
I20260925 11:01:16.881947 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.856288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.24.159.254:36795)
0925 11:01:16.856462 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:16.856467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:16.867080 (+ 10613us) server_negotiation.cc:407] Connection header received
0925 11:01:16.867389 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:16.867395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:16.867493 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:16.867596 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:16.868845 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.869908 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.871849 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.872453 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.872540 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.875379 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:16.875385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:16.875389 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:16.875423 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:16.878907 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.879777 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.879782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.879785 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.879919 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.880341 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.880346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.880350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.880703 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:16.880816 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:16.881405 (+ 589us) server_negotiation.cc:299] Negotiation successful
0925 11:01:16.881619 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:01:16.881964 31916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.855839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.24.159.253:43627)
0925 11:01:16.856049 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:16.856058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:16.866773 (+ 10715us) server_negotiation.cc:407] Connection header received
0925 11:01:16.866962 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:16.866965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:16.867039 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:16.867142 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:16.868527 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.869660 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.871677 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.872315 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.872412 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.875359 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:16.875365 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:16.875370 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:16.875400 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:16.879015 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.879763 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.879768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.879771 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.879919 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.880342 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.880346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.880350 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.880695 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:16.880818 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:16.881360 (+ 542us) server_negotiation.cc:299] Negotiation successful
0925 11:01:16.881585 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for oryx/127.24.159.252@KRBTEST.COM; defaulting to no policy
I20260925 11:01:16.882908 31937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.856173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.0.0.1:43156)
0925 11:01:16.866637 (+ 10464us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:16.866660 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:16.866833 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:16.867189 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:16.867194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:16.867888 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:16.868362 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.868374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.869855 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.869860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.871535 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.871549 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.872699 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.872706 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.872871 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.873958 (+ 1087us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:16.873976 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:16.875183 (+ 1207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:16.879130 (+ 3947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.879133 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.879136 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.879659 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.880046 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.880051 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.880054 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.880254 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.880878 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:16.880883 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:16.881106 (+ 223us) client_negotiation.cc:769] Sending connection context
0925 11:01:16.882223 (+ 1117us) client_negotiation.cc:240] Negotiation successful
0925 11:01:16.882492 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10339,"mutex_wait_us":42,"spinlock_wait_cycles":39680,"thread_start_us":10142,"threads_started":1}
I20260925 11:01:16.883658 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.855934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:55454)
0925 11:01:16.866958 (+ 11024us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:16.866998 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:16.867208 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:16.867638 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:16.867643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:16.868353 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:16.868671 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.868685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.870109 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.870114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.871713 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.871724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.872798 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.872805 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.872946 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.873874 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:16.873901 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:16.875183 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:16.879053 (+ 3870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.879058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.879062 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.879653 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.880046 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.880050 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.880054 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.880217 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.880879 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:16.880883 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:16.881148 (+ 265us) client_negotiation.cc:769] Sending connection context
0925 11:01:16.882721 (+ 1573us) client_negotiation.cc:240] Negotiation successful
0925 11:01:16.883055 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10027,"mutex_wait_us":51,"spinlock_wait_cycles":36224,"thread_start_us":9882,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.252@KRBTEST.COM" created.
Sep 25 11:01:16 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.254@KRBTEST.COM for oryx/127.24.159.253@KRBTEST.COM
I20260925 11:01:16.904879 31916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.891100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43160 (local address 127.24.159.253:43627)
0925 11:01:16.891262 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:16.891268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:16.891694 (+ 426us) server_negotiation.cc:407] Connection header received
0925 11:01:16.891957 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:16.891963 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:16.892070 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:16.892160 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:16.893263 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.894047 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.895585 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.896175 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.896304 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.899563 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:16.899568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:16.899574 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:16.899605 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:16.902154 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.902926 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.902930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.902932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.903064 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.903411 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.903414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.903416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.903695 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:16.903823 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:16.904305 (+ 482us) server_negotiation.cc:299] Negotiation successful
0925 11:01:16.904524 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 11:01:16.904938 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.890282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.0.0.1:43160)
0925 11:01:16.891579 (+ 1297us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:16.891611 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:16.891774 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:16.892208 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:16.892215 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:16.892846 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:16.893069 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.893082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.894219 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.894225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.895411 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.895423 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.896351 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.896357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.896549 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.897134 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:16.897151 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:16.899383 (+ 2232us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:16.902351 (+ 2968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.902364 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.902373 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.902804 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.903167 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.903172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.903176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.903302 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.903821 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:16.903830 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:16.904009 (+ 179us) client_negotiation.cc:769] Sending connection context
0925 11:01:16.904165 (+ 156us) client_negotiation.cc:240] Negotiation successful
0925 11:01:16.904516 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"thread_start_us":969,"threads_started":1}
I20260925 11:01:16.907142 31385 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.159.253:43627: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e"
member_type: NON_VOTER
last_known_addr {
host: "127.24.159.253"
port: 43627
}
attrs {
promote: true
}
}
dest_uuid: "1b678ee8f48245f7aaf9d8e30f224adb"
cas_config_opid_index: -1
I20260925 11:01:16.908354 31385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.252.keytab.
Entry for principal oryx/127.24.159.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.252.keytab.
I20260925 11:01:16.912070 31943 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ddf176fd1cc24080b1c99912aa360f1e (127.24.159.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } } }
I20260925 11:01:16.912981 31888 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:16.915196 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.252:41255
--webserver_interface=127.24.159.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.252.keytab
--principal=oryx/127.24.159.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:36795,127.24.159.253:43627,127.24.159.252:41255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:01:16.916167 31943 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.159.253:43627
I20260925 11:01:16.916347 31888 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Refusing update from remote peer 1b678ee8f48245f7aaf9d8e30f224adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260925 11:01:16.916864 31846 master.cc:537] Master@127.24.159.253:43627 shutting down...
I20260925 11:01:16.922557 31942 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:16.923771 31944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } } }
I20260925 11:01:16.924094 31944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:16.927982 31846 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:16.928416 31846 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:16.928664 31846 tablet_replica.cc:330] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: stopping tablet replica
W20260925 11:01:16.928669 31370 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260925 11:01:16.930953 31370 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb -> Peer ddf176fd1cc24080b1c99912aa360f1e (127.24.159.253:43627): Couldn't send request to peer ddf176fd1cc24080b1c99912aa360f1e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260925 11:01:16.945868 31846 master.cc:552] Master@127.24.159.253:43627 shutdown complete.
I20260925 11:01:16.945986 31846 master_runner.cc:305] Clearing existing system tablet
I20260925 11:01:16.946466 31846 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:16.946566 31846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:16.949560 31846 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260925 11:01:16.951273 31949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:16.951642 31846 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:16.951939 31846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 11:01:16.952028 31846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal
uuid: "ddf176fd1cc24080b1c99912aa360f1e"
format_stamp: "Formatted at 2026-09-25 11:01:16 on dist-test-slave-trms"
server_key: "172129c0f7b5640551d9910776112da0"
server_key_iv: "158d6a6e7a55bec3ebb0e1ee19a9cb17"
server_key_version: "encryptionkey@0"
I20260925 11:01:16.956918 31846 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:16.963620 31846 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 11:01:16.963798 31846 log.cc:1196] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260925 11:01:16.964213 31846 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: Deleting consensus metadata
I20260925 11:01:16.964462 31846 master_runner.cc:315] Copying system tablet from 127.24.159.254:36795
I20260925 11:01:16.966995 31846 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: tablet copy: Beginning tablet copy session from remote peer at address 127.24.159.254:36795
I20260925 11:01:16.984553 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.967776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55468 (local address 127.24.159.254:36795)
0925 11:01:16.967938 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:16.967943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:16.970789 (+ 2846us) server_negotiation.cc:407] Connection header received
0925 11:01:16.971011 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:16.971017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:16.971114 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:16.971216 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:16.972363 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.973486 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.975346 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.975964 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.976102 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.978338 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:16.978345 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:16.978350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:16.978388 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:16.981642 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.982440 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.982445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.982448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.982608 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:16.983026 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:16.983030 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:16.983033 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:16.983391 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:16.983534 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:16.983982 (+ 448us) server_negotiation.cc:299] Negotiation successful
0925 11:01:16.984205 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 11:01:16.984592 31955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:16.967692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:55468)
0925 11:01:16.970623 (+ 2931us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:16.970651 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:16.970848 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:16.971262 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:16.971267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:16.971862 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:16.972192 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.972207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.973661 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.973667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.975175 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:16.975190 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:16.976139 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:16.976146 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:16.976327 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:16.977023 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:16.977047 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:16.978087 (+ 1040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:16.981784 (+ 3697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.981789 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.981794 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.982330 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.982719 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:16.982724 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:16.982728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:16.982902 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:16.983507 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:16.983512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:16.983689 (+ 177us) client_negotiation.cc:769] Sending connection context
0925 11:01:16.983843 (+ 154us) client_negotiation.cc:240] Negotiation successful
0925 11:01:16.984153 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2798,"thread_start_us":2685,"threads_started":1}
I20260925 11:01:16.985859 31405 tablet_copy_service.cc:140] P 1b678ee8f48245f7aaf9d8e30f224adb: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ddf176fd1cc24080b1c99912aa360f1e ({username='oryx', principal='oryx/127.24.159.253@KRBTEST.COM'} at 127.0.0.1:55468)
I20260925 11:01:16.986172 31405 tablet_copy_service.cc:161] P 1b678ee8f48245f7aaf9d8e30f224adb: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ddf176fd1cc24080b1c99912aa360f1e at {username='oryx', principal='oryx/127.24.159.253@KRBTEST.COM'} at 127.0.0.1:55468: session id = ddf176fd1cc24080b1c99912aa360f1e-00000000000000000000000000000000
I20260925 11:01:16.990598 31405 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 11:01:16.992779 31846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 11:01:16.999780 31846 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: tablet copy: Starting download of 0 data blocks...
I20260925 11:01:17.000102 31846 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: tablet copy: Starting download of 1 WAL segments...
I20260925 11:01:17.002923 31846 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 11:01:17.007282 31405 tablet_copy_service.cc:342] P 1b678ee8f48245f7aaf9d8e30f224adb: Request end of tablet copy session ddf176fd1cc24080b1c99912aa360f1e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.159.253@KRBTEST.COM'} at 127.0.0.1:55468
I20260925 11:01:17.007555 31405 tablet_copy_service.cc:433] P 1b678ee8f48245f7aaf9d8e30f224adb: ending tablet copy session ddf176fd1cc24080b1c99912aa360f1e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ddf176fd1cc24080b1c99912aa360f1e
I20260925 11:01:17.018720 31846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:17.021369 31956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:17.022596 31957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:17.029721 31959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:17.035490 31846 server_base.cc:1034] running on GCE node
I20260925 11:01:17.035835 31846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:17.036499 31846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:17.037709 31846 hybrid_clock.cc:648] HybridClock initialized: now 1790334077037649 us; error 39 us; skew 500 ppm
I20260925 11:01:17.037942 31846 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:17 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:17.040206 31846 init.cc:377] Logged in from keytab as oryx/127.24.159.253@KRBTEST.COM (short username oryx)
I20260925 11:01:17.044996 31846 webserver.cc:460] Webserver started at http://127.24.159.253:33435/ using document root <none> and password file <none>
I20260925 11:01:17.045352 31846 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:17.045431 31846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:17.048682 31846 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260925 11:01:17.050606 31966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:17.050995 31846 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260925 11:01:17.051177 31846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal
uuid: "ddf176fd1cc24080b1c99912aa360f1e"
format_stamp: "Formatted at 2026-09-25 11:01:16 on dist-test-slave-trms"
server_key: "172129c0f7b5640551d9910776112da0"
server_key_iv: "158d6a6e7a55bec3ebb0e1ee19a9cb17"
server_key_version: "encryptionkey@0"
I20260925 11:01:17.051391 31846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:17.090710 31846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:17.094657 31846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:17.119112 31846 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.253:43627
I20260925 11:01:17.119158 32017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.253:43627 every 8 connection(s)
I20260925 11:01:17.120136 31846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
I20260925 11:01:17.125752 32018 sys_catalog.cc:263] Verifying existing consensus state
I20260925 11:01:17.129561 32018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: Bootstrap starting.
I20260925 11:01:17.146916 32018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 11:01:17.147480 32018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: Bootstrap complete.
I20260925 11:01:17.148622 32018 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } }
I20260925 11:01:17.148937 32018 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ddf176fd1cc24080b1c99912aa360f1e, State: Initialized, Role: LEARNER
I20260925 11:01:17.149221 32018 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } }
I20260925 11:01:17.150179 32021 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } } }
I20260925 11:01:17.150449 32018 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:17.150605 32021 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: This master's current role is: LEARNER
I20260925 11:01:17.162940 32032 catalog_manager.cc:1259] Loaded cluster ID: d15286d5e2564bee8585b41724319932
I20260925 11:01:17.163045 32032 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: loading cluster ID for follower catalog manager: success
I20260925 11:01:17.166465 32032 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: acquiring CA information for follower catalog manager: success
I20260925 11:01:17.168987 32032 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 11:01:17.287549 31945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:17.287843 31945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:17.287971 31945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:17.295569 31945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:17.295676 31945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:17.295753 31945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:17.295835 31945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:17.295892 31945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:17.305629 31945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:36795,127.24.159.253:43627,127.24.159.252:41255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.252.keytab
--principal=oryx/127.24.159.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.252:41255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.159.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:17.306972 31945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:17.308291 31945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 11:01:17.319005 31841 heartbeater.cc:503] Master 127.24.159.254:36795 was elected leader, sending a full tablet report...
W20260925 11:01:17.320015 32038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:17.320590 32037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:17.320851 32040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:17.321120 31945 server_base.cc:1034] running on GCE node
I20260925 11:01:17.321871 31945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:17.323261 31945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:17.324455 31945 hybrid_clock.cc:648] HybridClock initialized: now 1790334077324406 us; error 42 us; skew 500 ppm
I20260925 11:01:17.324776 31945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:17 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:17.327708 31945 init.cc:377] Logged in from keytab as oryx/127.24.159.252@KRBTEST.COM (short username oryx)
I20260925 11:01:17.329052 31945 webserver.cc:460] Webserver started at http://127.24.159.252:42215/ using document root <none> and password file <none>
I20260925 11:01:17.329685 31945 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:17.329790 31945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:17.330055 31945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:17.332983 31945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data/instance:
uuid: "8553047ecb914b2584d8cc730af9ad6a"
format_stamp: "Formatted at 2026-09-25 11:01:17 on dist-test-slave-trms"
server_key: "830d01aa2a1e1094785585709929faff"
server_key_iv: "40790981579619592a5e71b5e51f444f"
server_key_version: "encryptionkey@0"
I20260925 11:01:17.333678 31945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal/instance:
uuid: "8553047ecb914b2584d8cc730af9ad6a"
format_stamp: "Formatted at 2026-09-25 11:01:17 on dist-test-slave-trms"
server_key: "830d01aa2a1e1094785585709929faff"
server_key_iv: "40790981579619592a5e71b5e51f444f"
server_key_version: "encryptionkey@0"
I20260925 11:01:17.338526 31945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260925 11:01:17.341699 32047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:17.343024 31945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:17.343178 31945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal
uuid: "8553047ecb914b2584d8cc730af9ad6a"
format_stamp: "Formatted at 2026-09-25 11:01:17 on dist-test-slave-trms"
server_key: "830d01aa2a1e1094785585709929faff"
server_key_iv: "40790981579619592a5e71b5e51f444f"
server_key_version: "encryptionkey@0"
I20260925 11:01:17.343343 31945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:17.376482 31945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:17.380524 31945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:17.380779 31945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:17.399156 31945 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.252:41255
I20260925 11:01:17.399173 32098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.252:41255 every 8 connection(s)
I20260925 11:01:17.400560 31945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
I20260925 11:01:17.404465 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 31945
I20260925 11:01:17.404732 32099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:17.414130 32099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:01:17 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.252@KRBTEST.COM
Sep 25 11:01:17 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.252@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:17.429023 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.406666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50848 (local address 127.24.159.252:41255)
0925 11:01:17.407430 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.407448 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.407493 (+ 45us) server_negotiation.cc:407] Connection header received
0925 11:01:17.409412 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.409436 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.409866 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.410275 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.412096 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.413362 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.415149 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.415813 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.415942 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.420835 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.420843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.420857 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.420895 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.424356 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.425125 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.425134 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.425144 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.425342 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.425724 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.425730 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.425733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.426463 (+ 730us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.426679 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.427121 (+ 442us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.427565 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":329,"threads_started":1}
I20260925 11:01:17.432363 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.416942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:55474)
0925 11:01:17.417701 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.417749 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:17.417915 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:17.418278 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:17.418286 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:17.419086 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:17.419389 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.419399 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.420521 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.420526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.421700 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.421712 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.422647 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.422653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.422832 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.423406 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:17.423428 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:17.426311 (+ 2883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:17.429768 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.429776 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.429783 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.430362 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.430740 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.430744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.430746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.430876 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.431453 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:17.431459 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:17.431600 (+ 141us) client_negotiation.cc:769] Sending connection context
0925 11:01:17.431807 (+ 207us) client_negotiation.cc:240] Negotiation successful
0925 11:01:17.432097 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":348,"threads_started":1}
I20260925 11:01:17.432360 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.417067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.24.159.254:36795)
0925 11:01:17.417294 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.417301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.417847 (+ 546us) server_negotiation.cc:407] Connection header received
0925 11:01:17.418034 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.418037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.418098 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.418171 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.419527 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.420383 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.421838 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.422471 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.422593 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.426492 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.426497 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.426500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.426540 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.429652 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.430479 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.430485 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.430489 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.430637 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.430982 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.430986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.430990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.431332 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.431483 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.431937 (+ 454us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.432144 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 11:01:17.433621 32099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:01:17 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.252@KRBTEST.COM for oryx/127.24.159.253@KRBTEST.COM
I20260925 11:01:17.445097 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.434352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.0.0.1:43170)
0925 11:01:17.434533 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.434549 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:17.434626 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:17.435080 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:17.435083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:17.435358 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:17.435507 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.435514 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.436781 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.436784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.437686 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.437695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.438340 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.438345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.438471 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.438936 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:17.438953 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:17.440848 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:17.443371 (+ 2523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.443375 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.443377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.443720 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.443992 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.443995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.443998 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.444105 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.444493 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:17.444496 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:17.444601 (+ 105us) client_negotiation.cc:769] Sending connection context
0925 11:01:17.444696 (+ 95us) client_negotiation.cc:240] Negotiation successful
0925 11:01:17.444885 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:01:17.445233 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.434382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.24.159.253:43627)
0925 11:01:17.434839 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.434843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.434859 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:01:17.434907 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.434911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.434972 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.435058 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.435611 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.436685 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.437798 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.438217 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.438322 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.441019 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.441024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.441027 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.441056 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.443258 (+ 2202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.443800 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.443803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.443805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.443919 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.444174 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.444177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.444179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.444415 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.444526 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.444759 (+ 233us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.444910 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":245,"threads_started":1}
I20260925 11:01:17.446209 32099 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:01:17 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.252@KRBTEST.COM for oryx/127.24.159.252@KRBTEST.COM
I20260925 11:01:17.457338 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.446885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:41255 (local address 127.0.0.1:50858)
0925 11:01:17.447015 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.447029 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:17.447105 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:17.447398 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:17.447401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:17.447725 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:17.447865 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.447872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.448852 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.448855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.449654 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.449663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.450290 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.450294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.450461 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.450906 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:17.450922 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:17.452795 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:17.455387 (+ 2592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.455393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.455397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.455739 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.456070 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.456074 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.456077 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.456220 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.456646 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:17.456649 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:17.456767 (+ 118us) client_negotiation.cc:769] Sending connection context
0925 11:01:17.456869 (+ 102us) client_negotiation.cc:240] Negotiation successful
0925 11:01:17.457054 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 11:01:17.457423 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.447034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50858 (local address 127.24.159.252:41255)
0925 11:01:17.447189 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.447194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.447207 (+ 13us) server_negotiation.cc:407] Connection header received
0925 11:01:17.447241 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.447245 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.447292 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.447357 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.447978 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.448751 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.449754 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.450146 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.450269 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.452957 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.452962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.452966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.452993 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.455220 (+ 2227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.455849 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.455852 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.455855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.455964 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.456303 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.456306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.456308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.456558 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.456657 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.456947 (+ 290us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.457178 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":60}
I20260925 11:01:17.461856 32099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Bootstrap starting.
I20260925 11:01:17.464948 32099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:17.466019 32099 log.cc:824] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:17.468652 32099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: No bootstrap required, opened a new log
I20260925 11:01:17.475379 32099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } }
I20260925 11:01:17.475777 32099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:17.475903 32099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8553047ecb914b2584d8cc730af9ad6a, State: Initialized, Role: FOLLOWER
I20260925 11:01:17.476418 32099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } }
I20260925 11:01:17.477174 32106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } } }
I20260925 11:01:17.477456 32106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:01:17.478114 32099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:17.483842 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 31446
W20260925 11:01:17.486841 32117 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:01:17.486963 32117 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:01:17.490949 31945 master_runner.cc:428] Detected that this master 8553047ecb914b2584d8cc730af9ad6a is joining an existing cluster
I20260925 11:01:17.491170 31945 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.159.252:41255
Sep 25 11:01:17 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.193@KRBTEST.COM
I20260925 11:01:17.506369 32119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.485089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.24.159.193:43673)
0925 11:01:17.486666 (+ 1577us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.486676 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.486703 (+ 27us) server_negotiation.cc:407] Connection header received
0925 11:01:17.486769 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.486776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.487021 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.487169 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.488672 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.490399 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.492298 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.493221 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.494010 (+ 789us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.498267 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.498283 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.498301 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.498343 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.502120 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.503022 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.503032 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.503035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.503184 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.503680 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.503686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.503689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.504116 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.505507 (+ 1391us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.505669 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.505934 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1402,"thread_start_us":1555,"threads_started":1}
I20260925 11:01:17.507825 31569 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:17.527608 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.507031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43184 (local address 127.24.159.253:43627)
0925 11:01:17.507219 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.507225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.518981 (+ 11756us) server_negotiation.cc:407] Connection header received
0925 11:01:17.519529 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.519536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.519652 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.519745 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:17.520501 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.521656 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.524365 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.525678 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.525827 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.526049 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.527135 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.527306 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 11:01:17.528451 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.506546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.0.0.1:43184)
0925 11:01:17.518835 (+ 12289us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.518860 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:17.519303 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:17.519987 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:17.519993 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:17.520023 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:17.520316 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.520328 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.521863 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.521869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.524074 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.524091 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.526682 (+ 2591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.526689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.526897 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.526970 (+ 73us) client_negotiation.cc:769] Sending connection context
0925 11:01:17.527876 (+ 906us) client_negotiation.cc:240] Negotiation successful
0925 11:01:17.528047 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12139,"thread_start_us":12448,"threads_started":1}
I20260925 11:01:17.529647 31993 raft_consensus.cc:1203] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260925 11:01:17.530501 32021 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b678ee8f48245f7aaf9d8e30f224adb. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } } }
I20260925 11:01:17.530771 32021 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: This master's current role is: LEARNER
I20260925 11:01:17.539234 32106 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:17.539566 32106 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } }
I20260925 11:01:17.541289 32106 leader_election.cc:288] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795), ddf176fd1cc24080b1c99912aa360f1e (127.24.159.253:43627)
I20260925 11:01:17.542409 31395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8553047ecb914b2584d8cc730af9ad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" is_pre_election: true
I20260925 11:01:17.542596 31993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8553047ecb914b2584d8cc730af9ad6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddf176fd1cc24080b1c99912aa360f1e" is_pre_election: true
I20260925 11:01:17.543541 32049 leader_election.cc:302] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8553047ecb914b2584d8cc730af9ad6a; no voters: 1b678ee8f48245f7aaf9d8e30f224adb, ddf176fd1cc24080b1c99912aa360f1e
I20260925 11:01:17.544318 32106 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260925 11:01:17.576783 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.536724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43196 (local address 127.24.159.253:43627)
0925 11:01:17.537074 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.537080 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.553877 (+ 16797us) server_negotiation.cc:407] Connection header received
0925 11:01:17.555266 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.555273 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.555367 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.555456 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.558842 (+ 3386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.560001 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.567020 (+ 7019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.567676 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.567781 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.570495 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.570502 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.570507 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.570548 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.573745 (+ 3197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.574538 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.574543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.574547 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.574699 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.575134 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.575139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.575143 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.575493 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.575608 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.576186 (+ 578us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.576421 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225}
I20260925 11:01:17.577839 32129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.536585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.0.0.1:43196)
0925 11:01:17.553735 (+ 17150us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.553766 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:17.554908 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:17.555750 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:17.555755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:17.556547 (+ 792us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:17.556927 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.556942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.560255 (+ 3313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.560262 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.566850 (+ 6588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.566866 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.568138 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.568145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.568384 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.569132 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:17.569154 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:17.570274 (+ 1120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:17.573888 (+ 3614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.573893 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.573896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.574378 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.574833 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.574838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.574842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.575012 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.575686 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:17.575691 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:17.575878 (+ 187us) client_negotiation.cc:769] Sending connection context
0925 11:01:17.577054 (+ 1176us) client_negotiation.cc:240] Negotiation successful
0925 11:01:17.577380 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16969,"thread_start_us":25646,"threads_started":1}
I20260925 11:01:17.586225 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.545602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50864 (local address 127.24.159.252:41255)
0925 11:01:17.545818 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.545825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.562633 (+ 16808us) server_negotiation.cc:407] Connection header received
0925 11:01:17.562765 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.562770 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.562839 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.562926 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.565220 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.566460 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.572892 (+ 6432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.573657 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.573755 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.579587 (+ 5832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.579593 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.579598 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.579630 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.583360 (+ 3730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.584091 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.584099 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.584103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.584242 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.584592 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.584597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.584600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.585028 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.585160 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.585628 (+ 468us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.585843 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 11:01:17.586647 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.536558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55482 (local address 127.24.159.254:36795)
0925 11:01:17.536924 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.536930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.562056 (+ 25126us) server_negotiation.cc:407] Connection header received
0925 11:01:17.563134 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.563140 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.563222 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.563289 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.569918 (+ 6629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.571003 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.574708 (+ 3705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.575407 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.575508 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.580325 (+ 4817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.580333 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.580338 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.580387 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.583753 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.584482 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.584488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.584491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.584641 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.585076 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.585082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.585085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.585425 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.585564 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.586091 (+ 527us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.586331 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223}
I20260925 11:01:17.587205 32127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.546287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:41255 (local address 127.0.0.1:50864)
0925 11:01:17.562533 (+ 16246us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.562556 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:17.562667 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:17.563077 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:17.563081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:17.563859 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:17.564192 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.564204 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.566674 (+ 2470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.566679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.569728 (+ 3049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.569742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.574048 (+ 4306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.574054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.574246 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.576549 (+ 2303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:17.576571 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:17.577652 (+ 1081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:17.583528 (+ 5876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.583533 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.583537 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.583985 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.584361 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.584365 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.584368 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.584498 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.585222 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:17.585226 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:17.585378 (+ 152us) client_negotiation.cc:769] Sending connection context
0925 11:01:17.586484 (+ 1106us) client_negotiation.cc:240] Negotiation successful
0925 11:01:17.586764 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16124,"thread_start_us":7614,"threads_started":1}
I20260925 11:01:17.587734 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.537422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:55482)
0925 11:01:17.561932 (+ 24510us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.561957 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:17.562983 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:17.563508 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:17.563512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:17.564714 (+ 1202us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:17.565040 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.565051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.571174 (+ 6123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.571179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.572734 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.572748 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.575766 (+ 3018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.575773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.575932 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.578205 (+ 2273us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:17.578229 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:17.579298 (+ 1069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:17.583901 (+ 4603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.583907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.583911 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.584341 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.584774 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.584778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.584781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.584952 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.585629 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:17.585633 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:17.585831 (+ 198us) client_negotiation.cc:769] Sending connection context
0925 11:01:17.586888 (+ 1057us) client_negotiation.cc:240] Negotiation successful
0925 11:01:17.587188 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24339,"thread_start_us":25285,"threads_started":1}
Sep 25 11:01:17 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.254@KRBTEST.COM for oryx/127.24.159.252@KRBTEST.COM
I20260925 11:01:17.605000 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.592672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50878 (local address 127.24.159.252:41255)
0925 11:01:17.592819 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.592825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.592952 (+ 127us) server_negotiation.cc:407] Connection header received
0925 11:01:17.593167 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.593171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.593234 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.593317 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.593986 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.594862 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.596149 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.596621 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.596728 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.599947 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.599952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.599956 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.599985 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.602566 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.603186 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.603191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.603193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.603322 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.603672 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.603677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.603680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.603947 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.604070 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.604462 (+ 392us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.604664 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260925 11:01:17.604992 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.592717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:41255 (local address 127.0.0.1:50878)
0925 11:01:17.592877 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.592894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:17.593013 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:17.593359 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:17.593364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:17.593723 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:17.593858 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.593869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.594986 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.594991 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.596010 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.596022 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.596765 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.596770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.597011 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.597643 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:17.597660 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:17.599766 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:17.602696 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.602702 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.602706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.603080 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.603424 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.603429 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.603433 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.603573 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.604056 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:17.604061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:17.604230 (+ 169us) client_negotiation.cc:769] Sending connection context
0925 11:01:17.604351 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 11:01:17.604646 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 11:01:17.606650 31385 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.159.252:41255: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a"
member_type: NON_VOTER
last_known_addr {
host: "127.24.159.252"
port: 41255
}
attrs {
promote: true
}
}
dest_uuid: "1b678ee8f48245f7aaf9d8e30f224adb"
cas_config_opid_index: 5
I20260925 11:01:17.607527 31385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } }
I20260925 11:01:17.609236 32120 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8553047ecb914b2584d8cc730af9ad6a (127.24.159.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.609851 31993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 LEARNER]: Refusing update from remote peer 1b678ee8f48245f7aaf9d8e30f224adb: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 11:01:17.611831 32074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:17.612761 32120 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.159.252:41255
I20260925 11:01:17.615898 32074 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Refusing update from remote peer 1b678ee8f48245f7aaf9d8e30f224adb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260925 11:01:17.622947 32130 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260925 11:01:17.624316 32120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.624596 32120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:17.632928 32130 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:17.640015 31993 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 8553047ecb914b2584d8cc730af9ad6a (127.24.159.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.642526 31945 master.cc:537] Master@127.24.159.252:41255 shutting down...
W20260925 11:01:17.642671 32053 tablet.cc:2369] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:01:17.646525 32021 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: NON_VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: true } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.646819 32021 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: This master's current role is: LEARNER
I20260925 11:01:17.649174 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 31446
I20260925 11:01:17.653841 32131 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: attempting to promote NON_VOTER ddf176fd1cc24080b1c99912aa360f1e to VOTER
I20260925 11:01:17.654620 32131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } }
I20260925 11:01:17.656960 31993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 LEARNER]: Refusing update from remote peer 1b678ee8f48245f7aaf9d8e30f224adb: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260925 11:01:17.657552 32120 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260925 11:01:17.660163 32120 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ddf176fd1cc24080b1c99912aa360f1e (127.24.159.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.663462 31945 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:17.663872 31945 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:17.664083 31945 tablet_replica.cc:330] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: stopping tablet replica
I20260925 11:01:17.664110 32132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.664475 32132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:17.665591 31993 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ddf176fd1cc24080b1c99912aa360f1e (127.24.159.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.669428 32120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.669667 32120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:17.670051 32021 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.670308 32021 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:01:17.672379 31945 master.cc:552] Master@127.24.159.252:41255 shutdown complete.
I20260925 11:01:17.672464 31945 master_runner.cc:305] Clearing existing system tablet
I20260925 11:01:17.672849 31945 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:17.672956 31945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:17.678622 31945 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:01:17.678606 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:43673
--local_ip_for_outbound_sockets=127.24.159.193
--tserver_master_addrs=127.24.159.254:36795
--webserver_port=34895
--webserver_interface=127.24.159.193
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab
--principal=oryx/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.159.254:36795,127.24.159.253:43627
--tserver_master_addrs=127.24.159.254:36795,127.24.159.253:43627,127.24.159.252:41255 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:01:17.678892 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.678406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:41255 (local address 127.0.0.1:50880)
0925 11:01:17.678565 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.678696 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:41255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 11:01:17.679263 31372 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb -> Peer 8553047ecb914b2584d8cc730af9ad6a (127.24.159.252:41255): Couldn't send request to peer 8553047ecb914b2584d8cc730af9ad6a. Status: Network error: Client connection negotiation failed: client connection to 127.24.159.252:41255: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 11:01:17.680203 32137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:17.680615 31945 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:17.680851 31945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 11:01:17.680958 31945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal
uuid: "8553047ecb914b2584d8cc730af9ad6a"
format_stamp: "Formatted at 2026-09-25 11:01:17 on dist-test-slave-trms"
server_key: "830d01aa2a1e1094785585709929faff"
server_key_iv: "40790981579619592a5e71b5e51f444f"
server_key_version: "encryptionkey@0"
I20260925 11:01:17.685885 31945 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:17.692790 31945 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260925 11:01:17.692951 31945 log.cc:1196] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260925 11:01:17.693404 31945 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Deleting consensus metadata
I20260925 11:01:17.693647 31945 master_runner.cc:315] Copying system tablet from 127.24.159.254:36795
I20260925 11:01:17.699258 31945 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: tablet copy: Beginning tablet copy session from remote peer at address 127.24.159.254:36795
I20260925 11:01:17.717104 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.700065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55486 (local address 127.24.159.254:36795)
0925 11:01:17.700303 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:17.700310 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:17.702726 (+ 2416us) server_negotiation.cc:407] Connection header received
0925 11:01:17.703068 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:17.703075 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:17.703155 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:17.703223 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:17.704460 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.705586 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.707521 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.708172 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.708283 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.710722 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:17.710729 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:17.710734 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:17.710792 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:17.714135 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.714912 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.714917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.714921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.715067 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:17.715506 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:17.715511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:17.715515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:17.715840 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:17.715973 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:17.716505 (+ 532us) server_negotiation.cc:299] Negotiation successful
0925 11:01:17.716746 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260925 11:01:17.718097 32144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:17.700060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:55486)
0925 11:01:17.702618 (+ 2558us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:17.702638 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:17.702838 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:17.703451 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:17.703456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:17.704006 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:17.704252 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.704263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.705775 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.705780 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.707368 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:17.707382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:17.708593 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:17.708599 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:17.708800 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:17.709472 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:17.709493 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:17.710524 (+ 1031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:17.714298 (+ 3774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.714303 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.714307 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.714780 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.715195 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:17.715200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:17.715204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:17.715378 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:17.716053 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:17.716058 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:17.716226 (+ 168us) client_negotiation.cc:769] Sending connection context
0925 11:01:17.717350 (+ 1124us) client_negotiation.cc:240] Negotiation successful
0925 11:01:17.717675 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2398,"thread_start_us":2264,"threads_started":1}
I20260925 11:01:17.718675 31405 tablet_copy_service.cc:140] P 1b678ee8f48245f7aaf9d8e30f224adb: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 8553047ecb914b2584d8cc730af9ad6a ({username='oryx', principal='oryx/127.24.159.252@KRBTEST.COM'} at 127.0.0.1:55486)
I20260925 11:01:17.718842 31405 tablet_copy_service.cc:161] P 1b678ee8f48245f7aaf9d8e30f224adb: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 8553047ecb914b2584d8cc730af9ad6a at {username='oryx', principal='oryx/127.24.159.252@KRBTEST.COM'} at 127.0.0.1:55486: session id = 8553047ecb914b2584d8cc730af9ad6a-00000000000000000000000000000000
I20260925 11:01:17.722518 31405 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: Tablet Copy: opened 0 blocks and 1 log segments
I20260925 11:01:17.724148 31945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260925 11:01:17.731315 31945 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: tablet copy: Starting download of 0 data blocks...
I20260925 11:01:17.731611 31945 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: tablet copy: Starting download of 1 WAL segments...
I20260925 11:01:17.734076 31945 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260925 11:01:17.738612 31405 tablet_copy_service.cc:342] P 1b678ee8f48245f7aaf9d8e30f224adb: Request end of tablet copy session 8553047ecb914b2584d8cc730af9ad6a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.159.252@KRBTEST.COM'} at 127.0.0.1:55486
I20260925 11:01:17.738762 31405 tablet_copy_service.cc:433] P 1b678ee8f48245f7aaf9d8e30f224adb: ending tablet copy session 8553047ecb914b2584d8cc730af9ad6a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 8553047ecb914b2584d8cc730af9ad6a
I20260925 11:01:17.750028 31945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:17.753616 32145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:17.767665 32146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:17.782285 32148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:17.782655 31945 server_base.cc:1034] running on GCE node
I20260925 11:01:17.783139 31945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:17.783887 31945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:17.785156 31945 hybrid_clock.cc:648] HybridClock initialized: now 1790334077785109 us; error 35 us; skew 500 ppm
I20260925 11:01:17.785473 31945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:17 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334077, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:17.787629 31945 init.cc:377] Logged in from keytab as oryx/127.24.159.252@KRBTEST.COM (short username oryx)
I20260925 11:01:17.793279 31945 webserver.cc:460] Webserver started at http://127.24.159.252:39253/ using document root <none> and password file <none>
I20260925 11:01:17.793684 31945 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:17.793818 31945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:17.796823 31945 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260925 11:01:17.798681 32155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:17.799041 31945 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260925 11:01:17.799183 31945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal
uuid: "8553047ecb914b2584d8cc730af9ad6a"
format_stamp: "Formatted at 2026-09-25 11:01:17 on dist-test-slave-trms"
server_key: "830d01aa2a1e1094785585709929faff"
server_key_iv: "40790981579619592a5e71b5e51f444f"
server_key_version: "encryptionkey@0"
I20260925 11:01:17.800017 31945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:17.824985 31945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:17.828979 31945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:17.864425 31945 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.252:41255
I20260925 11:01:17.864503 32206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.252:41255 every 8 connection(s)
I20260925 11:01:17.865556 31945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
I20260925 11:01:17.870972 32207 sys_catalog.cc:263] Verifying existing consensus state
I20260925 11:01:17.874800 32207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Bootstrap starting.
I20260925 11:01:17.894515 32207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 11:01:17.895030 32207 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Bootstrap complete.
I20260925 11:01:17.896098 32207 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } }
I20260925 11:01:17.896412 32207 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 8553047ecb914b2584d8cc730af9ad6a, State: Initialized, Role: LEARNER
I20260925 11:01:17.896661 32207 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } }
I20260925 11:01:17.897493 32210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:17.897842 32207 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:17.897995 32210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: This master's current role is: LEARNER
I20260925 11:01:17.912475 32221 catalog_manager.cc:1259] Loaded cluster ID: d15286d5e2564bee8585b41724319932
I20260925 11:01:17.912591 32221 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: loading cluster ID for follower catalog manager: success
I20260925 11:01:17.916169 32221 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: acquiring CA information for follower catalog manager: success
I20260925 11:01:17.918721 32221 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 11:01:18.034286 32138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:18.034569 32138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:18.034665 32138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:18.041985 32138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:18.042100 32138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:18.042254 32138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:18.052305 32138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab
--principal=oryx/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:43673
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=34895
--tserver_master_addrs=127.24.159.254:36795,127.24.159.253:43627,127.24.159.252:41255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:18.053658 32138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:18.055249 32138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 11:01:18.060148 32122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.054400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:41255 (local address 127.0.0.1:50882)
0925 11:01:18.054610 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:18.054629 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:18.054774 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:18.055564 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:18.055568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:18.055597 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:18.055758 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.055768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.057029 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.057035 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.058260 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.058276 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.059360 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.059366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.059587 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.059622 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 11:01:18.059711 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 11:01:18.059855 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 11:01:18.060168 32225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.054488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50882 (local address 127.24.159.252:41255)
0925 11:01:18.055239 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:18.055244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:18.055263 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:01:18.055322 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:18.055327 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:18.055415 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:18.055527 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:18.055891 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.056898 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.058402 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.059199 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.059367 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.059556 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:18.059742 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 11:01:18.059867 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":524,"threads_started":1}
I20260925 11:01:18.061201 32182 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260925 11:01:18.061792 32210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b678ee8f48245f7aaf9d8e30f224adb. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:18.062292 32210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: This master's current role is: LEARNER
W20260925 11:01:18.068759 32227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:18.068789 32228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:18.068717 32230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:18.069537 32138 server_base.cc:1034] running on GCE node
I20260925 11:01:18.070072 32138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:18.070976 32138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:18.072160 32138 hybrid_clock.cc:648] HybridClock initialized: now 1790334078072110 us; error 41 us; skew 500 ppm
I20260925 11:01:18.072463 32138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:18 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:18.075546 32138 init.cc:377] Logged in from keytab as oryx/127.24.159.193@KRBTEST.COM (short username oryx)
I20260925 11:01:18.082487 32138 webserver.cc:460] Webserver started at http://127.24.159.193:34895/ using document root <none> and password file <none>
I20260925 11:01:18.083277 32138 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:18.083438 32138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:18.089973 32138 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 11:01:18.093545 32237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:18.095125 32138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:18.095279 32138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "6c8452f139574a7985ec962338123efd"
format_stamp: "Formatted at 2026-09-25 11:01:15 on dist-test-slave-trms"
server_key: "de2c4eab1acbb7ad38de53b7f5e2a7cc"
server_key_iv: "d8d0a29fcd2d6e1656f65f2c310b0c23"
server_key_version: "encryptionkey@0"
I20260925 11:01:18.096241 32138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:18.108402 32138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:18.112403 32138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:18.112651 32138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:18.113703 32138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:18.115067 32138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:18.115149 32138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:18.115250 32138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:18.115324 32138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:18.150985 32138 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:43673
I20260925 11:01:18.151068 32349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:43673 every 8 connection(s)
I20260925 11:01:18.152559 32138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:18.154584 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 32138
I20260925 11:01:18.154937 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 31580
Sep 25 11:01:18 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.252@KRBTEST.COM
Sep 25 11:01:18 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.253@KRBTEST.COM
Sep 25 11:01:18 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.194@KRBTEST.COM
Sep 25 11:01:18 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:18.192641 31576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.163911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:37071 (local address 127.24.159.254:36795)
0925 11:01:18.164107 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:18.164113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:18.175802 (+ 11689us) server_negotiation.cc:407] Connection header received
0925 11:01:18.175918 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:18.175924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:18.176003 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:18.176096 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:18.177297 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.178416 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.180297 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.180916 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.181009 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.186336 (+ 5327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:18.186343 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:18.186348 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:18.186391 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:18.189821 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.190495 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.190501 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.190504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.190652 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.191053 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.191058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.191060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.191421 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:18.191536 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:18.192065 (+ 529us) server_negotiation.cc:299] Negotiation successful
0925 11:01:18.192293 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 11:01:18.192896 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.157821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:35223 (local address 127.24.159.253:43627)
0925 11:01:18.158448 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:18.158455 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:18.166090 (+ 7635us) server_negotiation.cc:407] Connection header received
0925 11:01:18.167061 (+ 971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:18.167068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:18.167168 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:18.167267 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:18.168620 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.169696 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.175918 (+ 6222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.176550 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.176649 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.183663 (+ 7014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:18.183671 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:18.183675 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:18.183716 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:18.189642 (+ 5926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.190902 (+ 1260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.190907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.190911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.191051 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.191446 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.191451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.191454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.191765 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:18.191900 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:18.192355 (+ 455us) server_negotiation.cc:299] Negotiation successful
0925 11:01:18.192561 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":416,"threads_started":1}
I20260925 11:01:18.193760 32225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.157582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:42155 (local address 127.24.159.252:41255)
0925 11:01:18.158116 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:18.158122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:18.158144 (+ 22us) server_negotiation.cc:407] Connection header received
0925 11:01:18.159971 (+ 1827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:18.159977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:18.160058 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:18.160153 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:18.162357 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.163539 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.168598 (+ 5059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.169244 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.169342 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.182369 (+ 13027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:18.182377 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:18.182382 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:18.182425 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:18.188824 (+ 6399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.189532 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.189538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.189541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.189658 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.190181 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.190186 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.190227 (+ 41us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.190598 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:18.190737 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:18.193227 (+ 2490us) server_negotiation.cc:299] Negotiation successful
0925 11:01:18.193454 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416}
I20260925 11:01:18.194262 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.157033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.24.159.193:35223)
0925 11:01:18.165976 (+ 8943us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:18.166000 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:18.166143 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:18.167284 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:18.167288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:18.168050 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:18.168463 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.168477 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.170606 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.170611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.172261 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.172277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.176982 (+ 4705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.176988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.177183 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.178230 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:18.178252 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:18.183436 (+ 5184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:18.189800 (+ 6364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.189804 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.189808 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.190759 (+ 951us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.191190 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.191194 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.191197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.191322 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.191970 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:18.191973 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:18.192101 (+ 128us) client_negotiation.cc:769] Sending connection context
0925 11:01:18.193119 (+ 1018us) client_negotiation.cc:240] Negotiation successful
0925 11:01:18.193361 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8801,"thread_start_us":8586,"threads_started":1}
I20260925 11:01:18.194291 32359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.163266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.193:37071)
0925 11:01:18.175678 (+ 12412us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:18.175697 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:18.175819 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:18.176263 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:18.176268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:18.176772 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:18.177118 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.177130 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.178599 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.178607 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.180136 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.180151 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.181285 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.181291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.181472 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.182021 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:18.182042 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:18.186087 (+ 4045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:18.189929 (+ 3842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.189933 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.189936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.190393 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.190777 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.190782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.190785 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.190936 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.191627 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:18.191631 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:18.191760 (+ 129us) client_negotiation.cc:769] Sending connection context
0925 11:01:18.192890 (+ 1130us) client_negotiation.cc:240] Negotiation successful
0925 11:01:18.193138 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12275,"thread_start_us":12121,"threads_started":1}
I20260925 11:01:18.195415 32356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.156149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:41255 (local address 127.24.159.193:42155)
0925 11:01:18.157191 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:18.157236 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:18.159661 (+ 2425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:18.160414 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:18.160428 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:18.161260 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:18.162144 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.162169 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.165952 (+ 3783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.165961 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.168427 (+ 2466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.168446 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.169685 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.169691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.169985 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.171097 (+ 1112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:18.171130 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:18.174272 (+ 3142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:18.188977 (+ 14705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.188986 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.189006 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.189415 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.189780 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.189785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.189788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.190056 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.190811 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:18.190825 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:18.191300 (+ 475us) client_negotiation.cc:769] Sending connection context
0925 11:01:18.191633 (+ 333us) client_negotiation.cc:240] Negotiation successful
0925 11:01:18.192166 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":2058,"threads_started":1}
I20260925 11:01:18.195986 32352 heartbeater.cc:347] Connected to a master server at 127.24.159.253:43627
I20260925 11:01:18.196012 32351 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36795
I20260925 11:01:18.196324 32352 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:18.196329 32351 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:18.196677 32350 heartbeater.cc:347] Connected to a master server at 127.24.159.252:41255
I20260925 11:01:18.196854 32350 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:18.197291 32351 heartbeater.cc:511] Master 127.24.159.254:36795 requested a full tablet report, sending...
I20260925 11:01:18.197319 32352 heartbeater.cc:511] Master 127.24.159.253:43627 requested a full tablet report, sending...
I20260925 11:01:18.197434 32350 heartbeater.cc:511] Master 127.24.159.252:41255 requested a full tablet report, sending...
I20260925 11:01:18.198232 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.156276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.24.159.194:34645)
0925 11:01:18.170631 (+ 14355us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:18.170641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:18.170671 (+ 30us) server_negotiation.cc:407] Connection header received
0925 11:01:18.170736 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:18.170745 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:18.171014 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:18.173310 (+ 2296us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:18.173564 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.175111 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.177661 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.178813 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.185762 (+ 6949us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.186062 (+ 300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:18.186075 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:18.186096 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:18.186133 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:18.192717 (+ 6584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.193610 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.193619 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.193622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.193758 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.194221 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.194225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.194228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.195079 (+ 851us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:18.197389 (+ 2310us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:18.197550 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 11:01:18.197817 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14181,"thread_start_us":13313,"threads_started":1}
I20260925 11:01:18.199156 31385 ts_manager.cc:194] Re-registered known tserver with Master: 6c8452f139574a7985ec962338123efd (127.24.159.193:43673)
I20260925 11:01:18.199687 32172 ts_manager.cc:194] Registered new tserver with Master: 6c8452f139574a7985ec962338123efd (127.24.159.193:43673)
I20260925 11:01:18.199704 31983 ts_manager.cc:194] Registered new tserver with Master: 6c8452f139574a7985ec962338123efd (127.24.159.193:43673)
I20260925 11:01:18.200196 31385 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:37071
I20260925 11:01:18.200863 31703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:18.282275 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 31580
I20260925 11:01:18.296232 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:34645
--local_ip_for_outbound_sockets=127.24.159.194
--tserver_master_addrs=127.24.159.254:36795
--webserver_port=38205
--webserver_interface=127.24.159.194
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab
--principal=oryx/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.159.254:36795,127.24.159.253:43627
--tserver_master_addrs=127.24.159.254:36795,127.24.159.253:43627,127.24.159.252:41255 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:01:18.523263 32130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: NON_VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: true } } }
I20260925 11:01:18.523597 32130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:18.524832 32363 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: attempting to promote NON_VOTER 8553047ecb914b2584d8cc730af9ad6a to VOTER
I20260925 11:01:18.525365 32363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } }
I20260925 11:01:18.527060 32182 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 LEARNER]: Refusing update from remote peer 1b678ee8f48245f7aaf9d8e30f224adb: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260925 11:01:18.527158 31993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Refusing update from remote peer 1b678ee8f48245f7aaf9d8e30f224adb: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260925 11:01:18.527626 32362 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 11:01:18.528010 32363 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260925 11:01:18.530236 32362 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8553047ecb914b2584d8cc730af9ad6a (127.24.159.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:18.532845 32362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:18.533129 32362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:18.533035 32363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:18.533258 32363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:18.533481 31993 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8553047ecb914b2584d8cc730af9ad6a (127.24.159.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:18.535097 32363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:18.535351 32363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:18.536180 32182 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 8553047ecb914b2584d8cc730af9ad6a (127.24.159.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:18.540199 32210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:18.540392 32210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:01:18.539677 32366 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:18.540840 32366 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: This master's current role is: FOLLOWER
W20260925 11:01:18.613343 32361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:18.613715 32361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:18.613833 32361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:18.620908 32361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:18.621012 32361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:18.621127 32361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:18.630638 32361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab
--principal=oryx/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:34645
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=38205
--tserver_master_addrs=127.24.159.254:36795,127.24.159.253:43627,127.24.159.252:41255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:18.631832 32361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:18.633033 32361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:18.645653 32371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:18.645605 32372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:18.645774 32374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:18.646461 32361 server_base.cc:1034] running on GCE node
I20260925 11:01:18.647104 32361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:18.648036 32361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:18.649262 32361 hybrid_clock.cc:648] HybridClock initialized: now 1790334078649201 us; error 50 us; skew 500 ppm
I20260925 11:01:18.649570 32361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:18 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:18.652992 32361 init.cc:377] Logged in from keytab as oryx/127.24.159.194@KRBTEST.COM (short username oryx)
I20260925 11:01:18.654495 32361 webserver.cc:460] Webserver started at http://127.24.159.194:38205/ using document root <none> and password file <none>
I20260925 11:01:18.655151 32361 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:18.655259 32361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:18.660594 32361 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 11:01:18.663362 32381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:18.664785 32361 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:18.664921 32361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "1275fb03f9e748f281bc34fcab9cc133"
format_stamp: "Formatted at 2026-09-25 11:01:15 on dist-test-slave-trms"
server_key: "d399e64744cde32d3843145f82b5a05f"
server_key_iv: "1b16f2a1008e80ba62f6813f0ef61d28"
server_key_version: "encryptionkey@0"
I20260925 11:01:18.665874 32361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:18.680856 32361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:18.684700 32361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:18.684940 32361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:18.685834 32361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:18.687122 32361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:18.687207 32361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:18.687284 32361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:18.687343 32361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:18.717432 32361 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:34645
I20260925 11:01:18.717509 32493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:34645 every 8 connection(s)
I20260925 11:01:18.718933 32361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:18.724033 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 32361
I20260925 11:01:18.724300 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 31713
Sep 25 11:01:18 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:18.748574 32499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.722462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:54289 (local address 127.24.159.254:36795)
0925 11:01:18.734544 (+ 12082us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:18.734549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:18.734565 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:01:18.734618 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:18.734621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:18.734681 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:18.734775 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:18.736411 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.737139 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.738851 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.739255 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.739363 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.743075 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:18.743080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:18.743084 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:18.743118 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:18.745649 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.746381 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.746385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.746388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.746501 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.746891 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.746895 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.746897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.747173 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:18.747301 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:18.747987 (+ 686us) server_negotiation.cc:299] Negotiation successful
0925 11:01:18.748196 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11981,"thread_start_us":11883,"threads_started":1}
I20260925 11:01:18.749657 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.722373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:54289)
0925 11:01:18.731160 (+ 8787us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:18.731208 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:18.732944 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:18.735053 (+ 2109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:18.735062 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:18.735614 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:18.736230 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.736249 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.737296 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.737300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.738706 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.738719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.739496 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.739504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.739764 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.740476 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:18.740505 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:18.742850 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:18.745793 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.745803 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.745821 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.746264 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.746598 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.746603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.746607 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.746781 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.747294 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:18.747302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:18.747633 (+ 331us) client_negotiation.cc:769] Sending connection context
0925 11:01:18.747881 (+ 248us) client_negotiation.cc:240] Negotiation successful
0925 11:01:18.748412 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8485,"thread_start_us":8270,"threads_started":1}
I20260925 11:01:18.751199 32495 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36795
I20260925 11:01:18.751529 32495 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:18.752543 32495 heartbeater.cc:511] Master 127.24.159.254:36795 requested a full tablet report, sending...
I20260925 11:01:18.754313 31385 ts_manager.cc:194] Re-registered known tserver with Master: 1275fb03f9e748f281bc34fcab9cc133 (127.24.159.194:34645)
I20260925 11:01:18.755447 31385 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:54289
Sep 25 11:01:18 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334078, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.253@KRBTEST.COM
Sep 25 11:01:18 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.195@KRBTEST.COM
I20260925 11:01:18.770692 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.723717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:41255 (local address 127.24.159.194:53287)
0925 11:01:18.759858 (+ 36141us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:18.759879 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:18.760038 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:18.764840 (+ 4802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:18.764845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:18.764870 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:18.765272 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.765284 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.766783 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.766788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.768610 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.768622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.769941 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.769948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.770087 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.770124 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 11:01:18.770240 (+ 116us) client_negotiation.cc:240] Negotiation successful
0925 11:01:18.770337 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36003,"thread_start_us":36618,"threads_started":1}
I20260925 11:01:18.770735 32498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.721629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:53287 (local address 127.24.159.252:41255)
0925 11:01:18.764564 (+ 42935us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:18.764569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:18.764584 (+ 15us) server_negotiation.cc:407] Connection header received
0925 11:01:18.764637 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:18.764640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:18.764694 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:18.764779 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:18.765500 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.766641 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.768793 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.769761 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.769991 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.770069 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:18.770268 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 11:01:18.770405 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42833,"thread_start_us":43238,"threads_started":1}
I20260925 11:01:18.771450 32494 heartbeater.cc:347] Connected to a master server at 127.24.159.252:41255
I20260925 11:01:18.771591 32494 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:18.772042 32494 heartbeater.cc:511] Master 127.24.159.252:41255 requested a full tablet report, sending...
I20260925 11:01:18.773239 32172 ts_manager.cc:194] Registered new tserver with Master: 1275fb03f9e748f281bc34fcab9cc133 (127.24.159.194:34645)
I20260925 11:01:18.776955 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.722883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.24.159.194:50289)
0925 11:01:18.750694 (+ 27811us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:18.750712 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:18.750874 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:18.751477 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:18.751482 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:18.751931 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:18.752191 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.752205 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.764933 (+ 12728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.764938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.766223 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:18.766246 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.766986 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.766992 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:18.767218 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.767846 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:18.767870 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:18.771195 (+ 3325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:18.774688 (+ 3493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.774693 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.774698 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.775073 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.775410 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:18.775414 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:18.775418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:18.775536 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:18.776023 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:18.776028 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:18.776168 (+ 140us) client_negotiation.cc:769] Sending connection context
0925 11:01:18.776274 (+ 106us) client_negotiation.cc:240] Negotiation successful
0925 11:01:18.776575 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27666,"thread_start_us":36636,"threads_started":1}
I20260925 11:01:18.776996 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.722970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:50289 (local address 127.24.159.253:43627)
0925 11:01:18.730583 (+ 7613us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:18.730591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:18.750782 (+ 20191us) server_negotiation.cc:407] Connection header received
0925 11:01:18.751015 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:18.751021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:18.751108 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:18.751197 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:18.752438 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.753566 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.766379 (+ 12813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.766814 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.766951 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.771407 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:18.771415 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:18.771420 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:18.771474 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:18.774572 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.775191 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.775197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.775201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.775313 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.775642 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.775647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.775651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.775913 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:18.776055 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:18.776404 (+ 349us) server_negotiation.cc:299] Negotiation successful
0925 11:01:18.776611 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7477,"thread_start_us":7375,"threads_started":1}
I20260925 11:01:18.777706 32496 heartbeater.cc:347] Connected to a master server at 127.24.159.253:43627
I20260925 11:01:18.777822 32496 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:18.778180 32496 heartbeater.cc:511] Master 127.24.159.253:43627 requested a full tablet report, sending...
I20260925 11:01:18.778399 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:18.725601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59528 (local address 127.24.159.195:43259)
0925 11:01:18.758258 (+ 32657us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:18.758268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:18.758299 (+ 31us) server_negotiation.cc:407] Connection header received
0925 11:01:18.758377 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:18.758386 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:18.758651 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:18.758814 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:18.760680 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.762331 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.764283 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:18.765103 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:18.765220 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:18.772405 (+ 7185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:18.772418 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:18.772432 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:18.772464 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:18.775391 (+ 2927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.776003 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.776010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.776013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.776122 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:18.776459 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:18.776464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:18.776467 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:18.776773 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:18.777025 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:18.777745 (+ 720us) server_negotiation.cc:299] Negotiation successful
0925 11:01:18.777998 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32450,"thread_start_us":33383,"threads_started":1}
I20260925 11:01:18.779260 31983 ts_manager.cc:194] Registered new tserver with Master: 1275fb03f9e748f281bc34fcab9cc133 (127.24.159.194:34645)
I20260925 11:01:18.779737 31836 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:18.860677 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 31713
I20260925 11:01:18.874045 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:43259
--local_ip_for_outbound_sockets=127.24.159.195
--tserver_master_addrs=127.24.159.254:36795
--webserver_port=35791
--webserver_interface=127.24.159.195
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab
--principal=oryx/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.159.254:36795,127.24.159.253:43627
--tserver_master_addrs=127.24.159.254:36795,127.24.159.253:43627,127.24.159.252:41255 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:19.191846 32507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:19.192122 32507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:19.192220 32507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:19.199534 32507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:19.199656 32507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:19.199791 32507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:19.203245 32351 heartbeater.cc:503] Master 127.24.159.254:36795 was elected leader, sending a full tablet report...
I20260925 11:01:19.209985 32507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab
--principal=oryx/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:43259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=35791
--tserver_master_addrs=127.24.159.254:36795,127.24.159.253:43627,127.24.159.252:41255
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:19.211265 32507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:19.212540 32507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:19.224742 32512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:19.224807 32515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:19.224785 32513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:19.225520 32507 server_base.cc:1034] running on GCE node
I20260925 11:01:19.226079 32507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:19.226996 32507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:19.228193 32507 hybrid_clock.cc:648] HybridClock initialized: now 1790334079228148 us; error 40 us; skew 500 ppm
I20260925 11:01:19.228502 32507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:19 dist-test-slave-trms krb5kdc[31340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334079, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:19.231472 32507 init.cc:377] Logged in from keytab as oryx/127.24.159.195@KRBTEST.COM (short username oryx)
I20260925 11:01:19.232856 32507 webserver.cc:460] Webserver started at http://127.24.159.195:35791/ using document root <none> and password file <none>
I20260925 11:01:19.233508 32507 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:19.233634 32507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:19.238803 32507 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:19.241496 32522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:19.242834 32507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:19.242995 32507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "9f288844d41a440d8db0965a2ed7e6d7"
format_stamp: "Formatted at 2026-09-25 11:01:16 on dist-test-slave-trms"
server_key: "683753a8a49dd3f918fb417e5679c989"
server_key_iv: "b255358759dec45a451a0c28aa1932a4"
server_key_version: "encryptionkey@0"
I20260925 11:01:19.243913 32507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:19.258757 32507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:19.262758 32507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:19.263023 32507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:19.263952 32507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:19.265344 32507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:19.265434 32507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:19.265534 32507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:19.265615 32507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:19.294152 32507 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:43259
I20260925 11:01:19.294248 32634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:43259 every 8 connection(s)
I20260925 11:01:19.295655 32507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:19.301147 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 32507
Sep 25 11:01:19 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334079, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.252@KRBTEST.COM
Sep 25 11:01:19 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334079, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:19.334760 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.298383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:44835 (local address 127.24.159.252:41255)
0925 11:01:19.308227 (+ 9844us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.308234 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.314699 (+ 6465us) server_negotiation.cc:407] Connection header received
0925 11:01:19.314771 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.314776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.314857 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.314963 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.316428 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.317549 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.320625 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.321289 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.321390 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.327088 (+ 5698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.327096 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.327101 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.327147 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.330682 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.331695 (+ 1013us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.331700 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.331703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.331851 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.332389 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.332394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.332398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.332831 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.332988 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.334079 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.334350 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9683,"thread_start_us":9610,"threads_started":1}
I20260925 11:01:19.337357 32642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.299049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:41255 (local address 127.24.159.195:44835)
0925 11:01:19.314438 (+ 15389us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:19.314458 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:19.314589 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:19.315282 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:19.315288 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:19.315911 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:19.316251 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.316264 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.317745 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.317751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.320427 (+ 2676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.320458 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.321736 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.321743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.322077 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.322844 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:19.322907 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:19.326762 (+ 3855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:19.330866 (+ 4104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:19.330877 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:19.330913 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:19.331567 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:19.331959 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:19.331965 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:19.331969 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:19.332250 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:19.333095 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:19.333105 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:19.333594 (+ 489us) client_negotiation.cc:769] Sending connection context
0925 11:01:19.335076 (+ 1482us) client_negotiation.cc:240] Negotiation successful
0925 11:01:19.335700 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15212,"mutex_wait_us":1004,"thread_start_us":9722,"threads_started":1}
I20260925 11:01:19.341925 32643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.299049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.195:59505)
0925 11:01:19.309108 (+ 10059us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:19.309170 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:19.313091 (+ 3921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:19.313724 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:19.313739 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:19.314754 (+ 1015us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:19.315713 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.315739 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.318846 (+ 3107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.318852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.320525 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.320540 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.322602 (+ 2062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.322608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.322781 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.323675 (+ 894us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:19.323696 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:19.327466 (+ 3770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:19.331995 (+ 4529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:19.332001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:19.332005 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:19.332533 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:19.333915 (+ 1382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:19.333921 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:19.333925 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:19.334085 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:19.340848 (+ 6763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:19.340854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:19.341037 (+ 183us) client_negotiation.cc:769] Sending connection context
0925 11:01:19.341196 (+ 159us) client_negotiation.cc:240] Negotiation successful
0925 11:01:19.341505 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9679,"mutex_wait_us":1,"thread_start_us":14104,"threads_started":1}
I20260925 11:01:19.342098 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.298384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:59505 (local address 127.24.159.254:36795)
0925 11:01:19.308559 (+ 10175us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.308566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.309425 (+ 859us) server_negotiation.cc:407] Connection header received
0925 11:01:19.313217 (+ 3792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.313222 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.313312 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.313420 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.315906 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.316958 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.320654 (+ 3696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.321370 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.321488 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.327669 (+ 6181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.327676 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.327682 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.327728 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.331687 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.333653 (+ 1966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.333656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.333659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.333795 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.334526 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.334531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.334534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.334886 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.335031 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.341388 (+ 6357us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.341665 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10063,"thread_start_us":9952,"threads_started":1}
I20260925 11:01:19.346002 32636 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36795
I20260925 11:01:19.346396 32636 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:19.347407 32636 heartbeater.cc:511] Master 127.24.159.254:36795 requested a full tablet report, sending...
I20260925 11:01:19.349246 31385 ts_manager.cc:194] Re-registered known tserver with Master: 9f288844d41a440d8db0965a2ed7e6d7 (127.24.159.195:43259)
I20260925 11:01:19.350488 31385 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:59505
Sep 25 11:01:19 dist-test-slave-trms krb5kdc[31340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334079, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.253@KRBTEST.COM
I20260925 11:01:19.357725 32635 heartbeater.cc:347] Connected to a master server at 127.24.159.252:41255
I20260925 11:01:19.357889 32635 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:19.358372 32635 heartbeater.cc:511] Master 127.24.159.252:41255 requested a full tablet report, sending...
I20260925 11:01:19.359545 32172 ts_manager.cc:194] Registered new tserver with Master: 9f288844d41a440d8db0965a2ed7e6d7 (127.24.159.195:43259)
I20260925 11:01:19.362434 32644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.299527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.24.159.195:49647)
0925 11:01:19.310064 (+ 10537us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:19.310088 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:19.312635 (+ 2547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:19.315900 (+ 3265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:19.315904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:19.316359 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:19.316587 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.316598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.348364 (+ 31766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.348371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.350000 (+ 1629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.350017 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.351498 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.351506 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.351755 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.352370 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:19.352396 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:19.355530 (+ 3134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:19.359576 (+ 4046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:19.359581 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:19.359585 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:19.360073 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:19.360561 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:19.360566 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:19.360570 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:19.360732 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:19.361415 (+ 683us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:19.361420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:19.361591 (+ 171us) client_negotiation.cc:769] Sending connection context
0925 11:01:19.361735 (+ 144us) client_negotiation.cc:240] Negotiation successful
0925 11:01:19.362039 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10408,"thread_start_us":9063,"threads_started":1}
I20260925 11:01:19.363406 32639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.298384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:49647 (local address 127.24.159.253:43627)
0925 11:01:19.314957 (+ 16573us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.314963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.314983 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:01:19.315040 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.315044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.315129 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.315235 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.326057 (+ 10822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.348078 (+ 22021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.350210 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.350977 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.351096 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.355778 (+ 4682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.355785 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.355789 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.355836 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.359378 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.360220 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.360226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.360235 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.360406 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.360863 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.360868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.360872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.361209 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.361346 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.362851 (+ 1505us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.363059 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16445,"thread_start_us":16923,"threads_started":1}
I20260925 11:01:19.364645 32637 heartbeater.cc:347] Connected to a master server at 127.24.159.253:43627
I20260925 11:01:19.364777 32637 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:19.365111 32637 heartbeater.cc:511] Master 127.24.159.253:43627 requested a full tablet report, sending...
I20260925 11:01:19.366860 31983 ts_manager.cc:194] Registered new tserver with Master: 9f288844d41a440d8db0965a2ed7e6d7 (127.24.159.195:43259)
I20260925 11:01:19.375176 32652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.317869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43212 (local address 127.24.159.253:43627)
0925 11:01:19.355129 (+ 37260us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.355136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.355166 (+ 30us) server_negotiation.cc:407] Connection header received
0925 11:01:19.355227 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.355232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.355320 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.359769 (+ 4449us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.360042 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.361405 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.363551 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.364273 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.364391 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.367854 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.367861 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.367865 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.367903 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.371557 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.373206 (+ 1649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.373211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.373214 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.373356 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.373690 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.373695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.373699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.374051 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.374294 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.374609 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.374803 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34289,"thread_start_us":34408,"threads_started":1}
I20260925 11:01:19.376353 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.334609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50888 (local address 127.24.159.252:41255)
0925 11:01:19.335006 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.335012 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.356340 (+ 21328us) server_negotiation.cc:407] Connection header received
0925 11:01:19.356490 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.356495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.356596 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.356702 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.361151 (+ 4449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.362170 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.364018 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.364797 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.364891 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.369962 (+ 5071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.369969 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.369974 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.370012 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.373048 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.374049 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.374055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.374058 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.374223 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.374666 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.374672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.374675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.375025 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.375225 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.375830 (+ 605us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.376041 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":22304,"threads_started":1}
I20260925 11:01:19.378314 32641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.317163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.24.159.254:36795)
0925 11:01:19.342453 (+ 25290us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.342460 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.342478 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:01:19.342532 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.342537 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.342619 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.342722 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.366864 (+ 24142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.367719 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.369252 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.369647 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.369769 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.372175 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.372180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.372183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.372216 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.375046 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.376240 (+ 1194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.376244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.376247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.376401 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.377090 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.377094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.377097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.377426 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.377601 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.377878 (+ 277us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.378036 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25180,"thread_start_us":48142,"threads_started":1}
I20260925 11:01:19.381783 31385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:01:19.388734 31385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:19.419354 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.406119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:43259 (local address 127.0.0.1:59530)
0925 11:01:19.409490 (+ 3371us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:19.409513 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:19.409662 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:19.410834 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:19.410839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:19.410868 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:19.411257 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.411268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.413508 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.413514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.416028 (+ 2514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.416045 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.418477 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.418484 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.418676 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.418715 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 11:01:19.418809 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 11:01:19.418977 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3229,"threads_started":1}
I20260925 11:01:19.420581 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.406580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59530 (local address 127.24.159.195:43259)
0925 11:01:19.408647 (+ 2067us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.408658 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.409801 (+ 1143us) server_negotiation.cc:407] Connection header received
0925 11:01:19.409926 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.409937 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.410289 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.410495 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:19.411517 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.413329 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.416241 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.417899 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.418099 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.418332 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.420055 (+ 1723us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.420252 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1893,"thread_start_us":1654,"threads_started":1}
I20260925 11:01:19.424749 32570 tablet_service.cc:1467] Processing CreateTablet for tablet 915e2dce2726498c87dfcc7a248c0005 (DEFAULT_TABLE table=test-table [id=5cb75936c16a4543905012d679123a2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:19.426170 32570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915e2dce2726498c87dfcc7a248c0005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:19.434310 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.409761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45778 (local address 127.24.159.194:34645)
0925 11:01:19.425765 (+ 16004us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.425773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.425805 (+ 32us) server_negotiation.cc:407] Connection header received
0925 11:01:19.425883 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.425890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.426150 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.426419 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:19.429150 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.430078 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.432245 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.433282 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.433430 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.433596 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.433831 (+ 235us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.434004 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15838,"thread_start_us":15749,"threads_started":1}
I20260925 11:01:19.434433 32665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.408767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:60788)
0925 11:01:19.419734 (+ 10967us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:19.419758 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:19.419870 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:19.427928 (+ 8058us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:19.427932 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:19.427957 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:19.428098 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.428105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.429739 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.429745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.431797 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.431813 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.433582 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.433588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.433778 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.433812 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 11:01:19.433876 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 11:01:19.434014 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10841,"mutex_wait_us":60,"thread_start_us":418,"threads_started":1}
I20260925 11:01:19.434602 32663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.409733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:34645 (local address 127.0.0.1:45778)
0925 11:01:19.420803 (+ 11070us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:19.420820 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:19.420927 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:19.428562 (+ 7635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:19.428566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:19.428587 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:19.428972 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.428985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.430238 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.430243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.432051 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.432065 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.433470 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.433476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.433651 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.433690 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 11:01:19.433773 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 11:01:19.433912 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10933,"mutex_wait_us":47,"thread_start_us":16989,"threads_started":1}
I20260925 11:01:19.435695 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.409008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.24.159.193:43673)
0925 11:01:19.427276 (+ 18268us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.427284 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.427312 (+ 28us) server_negotiation.cc:407] Connection header received
0925 11:01:19.427406 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.427414 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.427723 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.427921 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:19.428265 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.429597 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.431995 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.433385 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.435034 (+ 1649us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.435197 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.435262 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.435417 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18096,"thread_start_us":17986,"threads_started":1}
I20260925 11:01:19.438411 32429 tablet_service.cc:1467] Processing CreateTablet for tablet 915e2dce2726498c87dfcc7a248c0005 (DEFAULT_TABLE table=test-table [id=5cb75936c16a4543905012d679123a2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:19.439278 32285 tablet_service.cc:1467] Processing CreateTablet for tablet 915e2dce2726498c87dfcc7a248c0005 (DEFAULT_TABLE table=test-table [id=5cb75936c16a4543905012d679123a2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:19.440250 32429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915e2dce2726498c87dfcc7a248c0005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:19.440698 32285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 915e2dce2726498c87dfcc7a248c0005. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:19.441123 32669 tablet_bootstrap.cc:492] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: Bootstrap starting.
I20260925 11:01:19.444264 32669 tablet_bootstrap.cc:654] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:19.445441 32669 log.cc:824] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:19.453424 32669 tablet_bootstrap.cc:492] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: No bootstrap required, opened a new log
I20260925 11:01:19.453944 32669 ts_tablet_manager.cc:1397] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260925 11:01:19.459226 32672 tablet_bootstrap.cc:492] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: Bootstrap starting.
I20260925 11:01:19.460065 32671 tablet_bootstrap.cc:492] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: Bootstrap starting.
I20260925 11:01:19.463006 32669 raft_consensus.cc:348] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } }
I20260925 11:01:19.463459 32669 raft_consensus.cc:374] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:19.463567 32669 raft_consensus.cc:729] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f288844d41a440d8db0965a2ed7e6d7, State: Initialized, Role: FOLLOWER
I20260925 11:01:19.463928 32671 tablet_bootstrap.cc:654] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:19.464249 32669 consensus_queue.cc:260] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } }
I20260925 11:01:19.465200 32672 tablet_bootstrap.cc:654] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:19.465248 32671 log.cc:824] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:19.466135 32669 ts_tablet_manager.cc:1428] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260925 11:01:19.466434 32636 heartbeater.cc:503] Master 127.24.159.254:36795 was elected leader, sending a full tablet report...
I20260925 11:01:19.469640 32672 log.cc:824] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:19.473658 32671 tablet_bootstrap.cc:492] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: No bootstrap required, opened a new log
I20260925 11:01:19.473842 32672 tablet_bootstrap.cc:492] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: No bootstrap required, opened a new log
I20260925 11:01:19.473919 32671 ts_tablet_manager.cc:1397] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.005s
I20260925 11:01:19.474095 32672 ts_tablet_manager.cc:1397] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s
I20260925 11:01:19.481262 32672 raft_consensus.cc:348] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } }
I20260925 11:01:19.481379 32671 raft_consensus.cc:348] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } }
I20260925 11:01:19.481627 32672 raft_consensus.cc:374] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:19.481709 32671 raft_consensus.cc:374] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:19.481719 32672 raft_consensus.cc:729] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8452f139574a7985ec962338123efd, State: Initialized, Role: FOLLOWER
I20260925 11:01:19.481788 32671 raft_consensus.cc:729] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1275fb03f9e748f281bc34fcab9cc133, State: Initialized, Role: FOLLOWER
I20260925 11:01:19.482328 32672 consensus_queue.cc:260] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } }
I20260925 11:01:19.482335 32671 consensus_queue.cc:260] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } }
I20260925 11:01:19.484063 32672 ts_tablet_manager.cc:1428] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260925 11:01:19.484351 32671 ts_tablet_manager.cc:1428] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260925 11:01:19.484601 32495 heartbeater.cc:503] Master 127.24.159.254:36795 was elected leader, sending a full tablet report...
I20260925 11:01:19.490628 32673 raft_consensus.cc:482] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:19.490899 32673 raft_consensus.cc:504] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } }
I20260925 11:01:19.492660 32673 leader_election.cc:288] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c8452f139574a7985ec962338123efd (127.24.159.193:43673), 1275fb03f9e748f281bc34fcab9cc133 (127.24.159.194:34645)
I20260925 11:01:19.498273 32666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.492982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:49835 (local address 127.24.159.193:43673)
0925 11:01:19.493142 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.493148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.493164 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:01:19.493231 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.493236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.493296 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.493363 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:19.493868 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.494784 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.496522 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.497362 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.497520 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.497573 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.497887 (+ 314us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.498013 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:01:19.498288 32644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.492845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.24.159.195:49835)
0925 11:01:19.492997 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:19.493016 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:19.493127 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:19.493532 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:19.493538 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:19.493565 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:19.493733 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.493745 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.494898 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.494904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.496383 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.496395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.497541 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.497548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.497748 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.497782 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 11:01:19.497866 (+ 84us) client_negotiation.cc:240] Negotiation successful
0925 11:01:19.498033 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260925 11:01:19.498674 32668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.493292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:39155 (local address 127.24.159.194:34645)
0925 11:01:19.493458 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.493464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.493479 (+ 15us) server_negotiation.cc:407] Connection header received
0925 11:01:19.493613 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.493617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.493692 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.493785 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:19.494144 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.495016 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.496767 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.497757 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.497927 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.497992 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.498340 (+ 348us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.498468 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:01:19.498710 32643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.493186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:34645 (local address 127.24.159.195:39155)
0925 11:01:19.493416 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:19.493431 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:19.493509 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:19.493813 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:19.493817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:19.493840 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:19.494023 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.494033 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.495133 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.495137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.496601 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:19.496614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.497923 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.497929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:19.498131 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.498170 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 11:01:19.498311 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 11:01:19.498467 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260925 11:01:19.499505 32449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "915e2dce2726498c87dfcc7a248c0005" candidate_uuid: "9f288844d41a440d8db0965a2ed7e6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1275fb03f9e748f281bc34fcab9cc133" is_pre_election: true
I20260925 11:01:19.499539 32305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "915e2dce2726498c87dfcc7a248c0005" candidate_uuid: "9f288844d41a440d8db0965a2ed7e6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8452f139574a7985ec962338123efd" is_pre_election: true
I20260925 11:01:19.499965 32449 raft_consensus.cc:2454] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f288844d41a440d8db0965a2ed7e6d7 in term 0.
I20260925 11:01:19.499959 32305 raft_consensus.cc:2454] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f288844d41a440d8db0965a2ed7e6d7 in term 0.
I20260925 11:01:19.500511 32523 leader_election.cc:302] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c8452f139574a7985ec962338123efd, 9f288844d41a440d8db0965a2ed7e6d7; no voters:
I20260925 11:01:19.500860 32673 raft_consensus.cc:2781] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:19.501024 32673 raft_consensus.cc:482] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:19.501127 32673 raft_consensus.cc:3037] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:19.503479 32673 raft_consensus.cc:504] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } }
I20260925 11:01:19.503999 32673 leader_election.cc:288] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [CANDIDATE]: Term 1 election: Requested vote from peers 6c8452f139574a7985ec962338123efd (127.24.159.193:43673), 1275fb03f9e748f281bc34fcab9cc133 (127.24.159.194:34645)
I20260925 11:01:19.504439 32305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "915e2dce2726498c87dfcc7a248c0005" candidate_uuid: "9f288844d41a440d8db0965a2ed7e6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8452f139574a7985ec962338123efd"
I20260925 11:01:19.504467 32449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "915e2dce2726498c87dfcc7a248c0005" candidate_uuid: "9f288844d41a440d8db0965a2ed7e6d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1275fb03f9e748f281bc34fcab9cc133"
I20260925 11:01:19.504642 32305 raft_consensus.cc:3037] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:19.504642 32449 raft_consensus.cc:3037] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:19.507807 32449 raft_consensus.cc:2454] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f288844d41a440d8db0965a2ed7e6d7 in term 1.
I20260925 11:01:19.507807 32305 raft_consensus.cc:2454] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f288844d41a440d8db0965a2ed7e6d7 in term 1.
I20260925 11:01:19.508180 32523 leader_election.cc:302] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c8452f139574a7985ec962338123efd, 9f288844d41a440d8db0965a2ed7e6d7; no voters:
I20260925 11:01:19.508459 32673 raft_consensus.cc:2781] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:19.509053 32673 raft_consensus.cc:686] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 1 LEADER]: Becoming Leader. State: Replica: 9f288844d41a440d8db0965a2ed7e6d7, State: Running, Role: LEADER
I20260925 11:01:19.509647 32673 consensus_queue.cc:237] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } }
I20260925 11:01:19.514366 31385 catalog_manager.cc:5495] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f288844d41a440d8db0965a2ed7e6d7 (127.24.159.195). New cstate: current_term: 1 leader_uuid: "9f288844d41a440d8db0965a2ed7e6d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } health_report { overall_health: UNKNOWN } } }
I20260925 11:01:19.539453 32664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.534401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59536 (local address 127.24.159.195:43259)
0925 11:01:19.534589 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.534595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.534613 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:01:19.534799 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.534802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.534876 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.535006 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:19.535343 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.536047 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.537344 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.537733 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.537839 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.538314 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:19.538842 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:19.538984 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.539079 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.539152 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:01:19.547374 32449 raft_consensus.cc:1261] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [term 1 FOLLOWER]: Refusing update from remote peer 9f288844d41a440d8db0965a2ed7e6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:19.547392 32305 raft_consensus.cc:1261] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [term 1 FOLLOWER]: Refusing update from remote peer 9f288844d41a440d8db0965a2ed7e6d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:19.548353 32678 consensus_queue.cc:1035] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:19.548717 32673 consensus_queue.cc:1035] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 11:01:19.554913 32638 tablet.cc:2369] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:01:19.561977 32681 mvcc.cc:205] Tried to move back new op lower bound from 7333208389816250368 to 7333208389675335680. Current Snapshot: MvccSnapshot[applied={T|T < 7333208389816250368}]
I20260925 11:01:19.567365 32683 mvcc.cc:205] Tried to move back new op lower bound from 7333208389816250368 to 7333208389675335680. Current Snapshot: MvccSnapshot[applied={T|T < 7333208389816250368}]
I20260925 11:01:19.584034 31385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:19.584275 31385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:19.591166 31385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1b678ee8f48245f7aaf9d8e30f224adb: Sending DeleteTablet for 3 replicas of tablet 915e2dce2726498c87dfcc7a248c0005
I20260925 11:01:19.592267 32570 tablet_service.cc:1514] Processing DeleteTablet for tablet 915e2dce2726498c87dfcc7a248c0005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:19 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:59530
I20260925 11:01:19.592545 32285 tablet_service.cc:1514] Processing DeleteTablet for tablet 915e2dce2726498c87dfcc7a248c0005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:19 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:60788
I20260925 11:01:19.593268 32429 tablet_service.cc:1514] Processing DeleteTablet for tablet 915e2dce2726498c87dfcc7a248c0005 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:19 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:45778
I20260925 11:01:19.603634 32692 tablet_replica.cc:330] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: stopping tablet replica
I20260925 11:01:19.604892 32692 raft_consensus.cc:2229] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:19.606380 32692 raft_consensus.cc:2258] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:19.607858 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 31353
I20260925 11:01:19.609033 31415 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:19.611092 32691 tablet_replica.cc:330] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: stopping tablet replica
I20260925 11:01:19.611676 32691 raft_consensus.cc:2229] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:19.612211 32691 raft_consensus.cc:2258] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:19.612345 32690 tablet_replica.cc:330] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: stopping tablet replica
I20260925 11:01:19.612814 32690 raft_consensus.cc:2229] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:01:19.614467 32691 ts_tablet_manager.cc:1905] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:19.615011 32692 ts_tablet_manager.cc:1905] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:19.613618 32690 raft_consensus.cc:2258] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:19.621507 32691 ts_tablet_manager.cc:1918] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:19.622017 32691 log.cc:1196] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/915e2dce2726498c87dfcc7a248c0005
I20260925 11:01:19.622570 32691 ts_tablet_manager.cc:1939] T 915e2dce2726498c87dfcc7a248c0005 P 6c8452f139574a7985ec962338123efd: Deleting consensus metadata
I20260925 11:01:19.623880 32690 ts_tablet_manager.cc:1905] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:19.627627 32652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.595391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43226 (local address 127.24.159.253:43627)
0925 11:01:19.595564 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.595569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.606120 (+ 10551us) server_negotiation.cc:407] Connection header received
0925 11:01:19.609085 (+ 2965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.609092 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.609176 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.609256 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.613812 (+ 4556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.614985 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.617064 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.617757 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.617877 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.621682 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.621689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.621693 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.621735 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.624581 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.625375 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.625381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.625385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.625534 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.625935 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.625940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.625944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.626318 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.626540 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.627095 (+ 555us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.627309 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 11:01:19.630410 32690 ts_tablet_manager.cc:1918] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:19.630585 32690 log.cc:1196] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/915e2dce2726498c87dfcc7a248c0005
I20260925 11:01:19.631142 32690 ts_tablet_manager.cc:1939] T 915e2dce2726498c87dfcc7a248c0005 P 9f288844d41a440d8db0965a2ed7e6d7: Deleting consensus metadata
I20260925 11:01:19.621507 32692 ts_tablet_manager.cc:1918] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:19.634505 32692 log.cc:1196] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/915e2dce2726498c87dfcc7a248c0005
I20260925 11:01:19.635075 32692 ts_tablet_manager.cc:1939] T 915e2dce2726498c87dfcc7a248c0005 P 1275fb03f9e748f281bc34fcab9cc133: Deleting consensus metadata
I20260925 11:01:19.641647 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.606714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.24.159.252:41255)
0925 11:01:19.606880 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.606885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.628547 (+ 21662us) server_negotiation.cc:407] Connection header received
0925 11:01:19.628816 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.628822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.628902 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.628968 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.630849 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.631893 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.633362 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.633760 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.633872 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.636881 (+ 3009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.636886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.636889 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.636932 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.639306 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.639853 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.639859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.639862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.639986 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.640381 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.640385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.640387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.640643 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.640851 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.641150 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.641356 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260925 11:01:19.777563 31369 catalog_manager.cc:4841] TS 1275fb03f9e748f281bc34fcab9cc133 (127.24.159.194:34645): tablet 915e2dce2726498c87dfcc7a248c0005 (table test-table [id=5cb75936c16a4543905012d679123a2e]) successfully deleted
I20260925 11:01:19.777963 31369 catalog_manager.cc:4841] TS 6c8452f139574a7985ec962338123efd (127.24.159.193:43673): tablet 915e2dce2726498c87dfcc7a248c0005 (table test-table [id=5cb75936c16a4543905012d679123a2e]) successfully deleted
I20260925 11:01:19.778160 31370 catalog_manager.cc:4841] TS 9f288844d41a440d8db0965a2ed7e6d7 (127.24.159.195:43259): tablet 915e2dce2726498c87dfcc7a248c0005 (table test-table [id=5cb75936c16a4543905012d679123a2e]) successfully deleted
I20260925 11:01:19.778438 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 31353
I20260925 11:01:19.823086 32640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.799872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.24.159.252:41255)
0925 11:01:19.800094 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.800101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.807663 (+ 7562us) server_negotiation.cc:407] Connection header received
0925 11:01:19.808029 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.808035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.808107 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.808167 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.810787 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.811906 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.813843 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.814435 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.814551 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.817375 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.817380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.817383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.817417 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.820270 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.820973 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.820978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.820982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.821128 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.821507 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.821512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.821515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.821867 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.822109 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.822522 (+ 413us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.822732 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 11:01:19.824775 32652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:19.800299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.24.159.253:43627)
0925 11:01:19.800485 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:19.800490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:19.807475 (+ 6985us) server_negotiation.cc:407] Connection header received
0925 11:01:19.807923 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:19.807929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:19.808030 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:19.808103 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:19.810787 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.811925 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.814825 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:19.815391 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:19.815523 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:19.818579 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:19.818585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:19.818589 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:19.818631 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:19.821419 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.822540 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.822545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.822548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.822687 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:19.823103 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:19.823109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:19.823112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:19.823478 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:19.823696 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:19.824211 (+ 515us) server_negotiation.cc:299] Negotiation successful
0925 11:01:19.824436 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 11:01:20.572342 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:20.571668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:33599)
0925 11:01:20.572069 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:20.572184 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":202,"threads_started":1}
W20260925 11:01:20.572741 32495 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:36795 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:20.576916 32703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:20.576122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.193:40879)
0925 11:01:20.576573 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:20.576707 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":214,"threads_started":1}
W20260925 11:01:20.583997 32351 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:36795 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:20.584493 32704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:20.577520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.195:47621)
0925 11:01:20.584123 (+ 6603us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:20.584288 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6477,"thread_start_us":6328,"threads_started":1}
W20260925 11:01:20.584841 32636 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:36795 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:21.091805 32706 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1b678ee8f48245f7aaf9d8e30f224adb)
I20260925 11:01:21.091814 32705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1b678ee8f48245f7aaf9d8e30f224adb)
I20260925 11:01:21.092013 32705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } }
I20260925 11:01:21.092077 32706 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } }
I20260925 11:01:21.092757 32705 leader_election.cc:288] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795), ddf176fd1cc24080b1c99912aa360f1e (127.24.159.253:43627)
I20260925 11:01:21.099797 32706 leader_election.cc:288] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795), 8553047ecb914b2584d8cc730af9ad6a (127.24.159.252:41255)
I20260925 11:01:21.106669 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.092855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40102)
0925 11:01:21.106288 (+ 13433us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.106484 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13294,"threads_started":1}
W20260925 11:01:21.108342 32156 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:21.114037 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.092966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43627 (local address 127.0.0.1:34506)
0925 11:01:21.106833 (+ 13867us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.106850 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:21.106957 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:21.107820 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:21.107826 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:21.107853 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:21.108089 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:21.108102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:21.109553 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:21.109558 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:21.111570 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:21.111584 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:21.113197 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:21.113202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:21.113367 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:21.113439 (+ 72us) client_negotiation.cc:769] Sending connection context
0925 11:01:21.113521 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 11:01:21.113657 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13786,"mutex_wait_us":25,"thread_start_us":13033,"threads_started":1}
I20260925 11:01:21.114596 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.107632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40104)
0925 11:01:21.114083 (+ 6451us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.114291 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6326,"threads_started":1}
I20260925 11:01:21.115119 32709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.093135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34506 (local address 127.24.159.253:43627)
0925 11:01:21.107557 (+ 14422us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:21.107561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:21.107576 (+ 15us) server_negotiation.cc:407] Connection header received
0925 11:01:21.107632 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:21.107636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:21.107688 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:21.107770 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:21.108287 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:21.109394 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:21.111766 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:21.113031 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:21.114491 (+ 1460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:21.114585 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:21.114630 (+ 45us) server_negotiation.cc:299] Negotiation successful
0925 11:01:21.114757 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14339,"thread_start_us":14180,"threads_started":1}
I20260925 11:01:21.115360 32710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.093682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:41255 (local address 127.0.0.1:57672)
0925 11:01:21.105394 (+ 11712us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.105419 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:21.105580 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:21.108636 (+ 3056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:21.108641 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:21.108668 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:21.109024 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:21.109036 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:21.110850 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:21.110856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:21.112899 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:21.112913 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:21.114579 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:21.114587 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:21.114764 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:21.114791 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 11:01:21.114854 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 11:01:21.114988 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11578,"thread_start_us":11990,"threads_started":1}
W20260925 11:01:21.115170 31967 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:21.115777 32711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.107406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.24.159.252:41255)
0925 11:01:21.108330 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:21.108336 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:21.108352 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:01:21.108398 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:21.108403 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:21.108495 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:21.109176 (+ 681us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:21.109410 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:21.110706 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:21.113107 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:21.114367 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:21.115259 (+ 892us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:21.115325 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:21.115373 (+ 48us) server_negotiation.cc:299] Negotiation successful
0925 11:01:21.115501 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":801,"thread_start_us":665,"threads_started":1}
I20260925 11:01:21.116364 32182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ddf176fd1cc24080b1c99912aa360f1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8553047ecb914b2584d8cc730af9ad6a" is_pre_election: true
I20260925 11:01:21.116730 32182 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ddf176fd1cc24080b1c99912aa360f1e in term 1.
I20260925 11:01:21.116689 31993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8553047ecb914b2584d8cc730af9ad6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ddf176fd1cc24080b1c99912aa360f1e" is_pre_election: true
I20260925 11:01:21.116816 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.116468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40108)
0925 11:01:21.116592 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.116671 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260925 11:01:21.116935 31993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8553047ecb914b2584d8cc730af9ad6a in term 1.
W20260925 11:01:21.117082 32156 leader_election.cc:334] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795): Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:21.117497 31970 leader_election.cc:302] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8553047ecb914b2584d8cc730af9ad6a, ddf176fd1cc24080b1c99912aa360f1e; no voters:
I20260925 11:01:21.117686 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.117335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40114)
0925 11:01:21.117474 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.117554 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 11:01:21.117681 32157 leader_election.cc:302] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8553047ecb914b2584d8cc730af9ad6a, ddf176fd1cc24080b1c99912aa360f1e; no voters: 1b678ee8f48245f7aaf9d8e30f224adb
I20260925 11:01:21.117874 32706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 11:01:21.117950 32705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260925 11:01:21.118042 32706 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1b678ee8f48245f7aaf9d8e30f224adb)
I20260925 11:01:21.118050 32705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1b678ee8f48245f7aaf9d8e30f224adb)
I20260925 11:01:21.118127 32705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 1 FOLLOWER]: Advancing to term 2
I20260925 11:01:21.118131 32706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 1 FOLLOWER]: Advancing to term 2
W20260925 11:01:21.117878 31967 leader_election.cc:334] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795): Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:21.120342 32705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } }
I20260925 11:01:21.120913 32705 leader_election.cc:288] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 2 election: Requested vote from peers 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795), ddf176fd1cc24080b1c99912aa360f1e (127.24.159.253:43627)
I20260925 11:01:21.121148 32706 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } }
I20260925 11:01:21.121382 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.121022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40116)
0925 11:01:21.121163 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.121261 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260925 11:01:21.121589 31993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8553047ecb914b2584d8cc730af9ad6a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ddf176fd1cc24080b1c99912aa360f1e"
I20260925 11:01:21.121861 31993 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 8553047ecb914b2584d8cc730af9ad6a in current term 2: Already voted for candidate ddf176fd1cc24080b1c99912aa360f1e in this term.
I20260925 11:01:21.122367 32706 leader_election.cc:288] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [CANDIDATE]: Term 2 election: Requested vote from peers 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795), 8553047ecb914b2584d8cc730af9ad6a (127.24.159.252:41255)
I20260925 11:01:21.122812 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.122486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40132)
0925 11:01:21.122610 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.122676 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260925 11:01:21.123353 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.122950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40144)
0925 11:01:21.123116 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.123200 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 11:01:21.123351 32182 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ddf176fd1cc24080b1c99912aa360f1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "8553047ecb914b2584d8cc730af9ad6a"
I20260925 11:01:21.123623 32182 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ddf176fd1cc24080b1c99912aa360f1e in current term 2: Already voted for candidate 8553047ecb914b2584d8cc730af9ad6a in this term.
I20260925 11:01:21.123769 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.123380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40158)
0925 11:01:21.123578 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.123650 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260925 11:01:21.123919 32156 leader_election.cc:334] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795): Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
W20260925 11:01:21.123936 31967 leader_election.cc:334] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795): Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:21.124038 32156 leader_election.cc:302] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8553047ecb914b2584d8cc730af9ad6a; no voters: 1b678ee8f48245f7aaf9d8e30f224adb, ddf176fd1cc24080b1c99912aa360f1e
I20260925 11:01:21.124285 32705 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 11:01:21.124598 31970 leader_election.cc:302] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ddf176fd1cc24080b1c99912aa360f1e; no voters: 1b678ee8f48245f7aaf9d8e30f224adb, 8553047ecb914b2584d8cc730af9ad6a
I20260925 11:01:21.124821 32706 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260925 11:01:21.574039 32715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.573350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:53767)
0925 11:01:21.573761 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.573844 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":212,"threads_started":1}
I20260925 11:01:21.585363 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.584687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.193:56301)
0925 11:01:21.585117 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.585203 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":204,"threads_started":1}
I20260925 11:01:21.586169 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:21.585409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.195:36171)
0925 11:01:21.585932 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:21.586017 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":327,"threads_started":1}
I20260925 11:01:22.575578 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.574866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:42301)
0925 11:01:22.575267 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.575386 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":199,"threads_started":1}
I20260925 11:01:22.587046 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.586148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.193:59861)
0925 11:01:22.586673 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.586781 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":294,"threads_started":1}
I20260925 11:01:22.587615 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.586956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.195:57981)
0925 11:01:22.587395 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.587470 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":243,"threads_started":1}
I20260925 11:01:22.627547 32721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:22.627748 32721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } }
I20260925 11:01:22.628407 32721 leader_election.cc:288] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795), ddf176fd1cc24080b1c99912aa360f1e (127.24.159.253:43627)
I20260925 11:01:22.628971 31993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8553047ecb914b2584d8cc730af9ad6a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ddf176fd1cc24080b1c99912aa360f1e" is_pre_election: true
I20260925 11:01:22.629189 31993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8553047ecb914b2584d8cc730af9ad6a in term 2.
I20260925 11:01:22.629366 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.628588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40386)
0925 11:01:22.629052 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.629172 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":223,"threads_started":1}
I20260925 11:01:22.629640 32157 leader_election.cc:302] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8553047ecb914b2584d8cc730af9ad6a, ddf176fd1cc24080b1c99912aa360f1e; no voters:
I20260925 11:01:22.629882 32721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260925 11:01:22.629993 32721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:22.630089 32721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 2 FOLLOWER]: Advancing to term 3
I20260925 11:01:22.630735 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.630365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40390)
0925 11:01:22.630524 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.630605 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 11:01:22.630965 32156 leader_election.cc:334] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795): Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:22.632336 32721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } }
I20260925 11:01:22.632856 32721 leader_election.cc:288] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 3 election: Requested vote from peers 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795), ddf176fd1cc24080b1c99912aa360f1e (127.24.159.253:43627)
I20260925 11:01:22.633340 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.632994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40400)
0925 11:01:22.633130 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.633222 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:01:22.633332 31993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8553047ecb914b2584d8cc730af9ad6a" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ddf176fd1cc24080b1c99912aa360f1e"
I20260925 11:01:22.633543 31993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 2 FOLLOWER]: Advancing to term 3
I20260925 11:01:22.634151 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.633845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40402)
0925 11:01:22.633997 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.634057 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 11:01:22.634349 32156 leader_election.cc:334] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795): Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
I20260925 11:01:22.635659 31993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8553047ecb914b2584d8cc730af9ad6a in term 3.
I20260925 11:01:22.636009 32157 leader_election.cc:302] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8553047ecb914b2584d8cc730af9ad6a, ddf176fd1cc24080b1c99912aa360f1e; no voters: 1b678ee8f48245f7aaf9d8e30f224adb
I20260925 11:01:22.636251 32721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 3 FOLLOWER]: Leader election won for term 3
I20260925 11:01:22.636606 32721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [term 3 LEADER]: Becoming Leader. State: Replica: 8553047ecb914b2584d8cc730af9ad6a, State: Running, Role: LEADER
I20260925 11:01:22.636955 32721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } }
I20260925 11:01:22.638697 32724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8553047ecb914b2584d8cc730af9ad6a. Latest consensus state: current_term: 3 leader_uuid: "8553047ecb914b2584d8cc730af9ad6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:22.639032 32724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:22.639566 32726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:22.642999 32726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:22.643777 32726 catalog_manager.cc:1259] Loaded cluster ID: d15286d5e2564bee8585b41724319932
I20260925 11:01:22.643846 32726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:22.644920 32726 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:22.645675 32726 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Loaded TSK: 0
I20260925 11:01:22.646556 32726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:01:22.686139 32171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57656:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:01:22.692830 32171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:22.696772 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.696397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40428)
0925 11:01:22.696553 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.696631 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260925 11:01:22.696841 31993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [term 3 FOLLOWER]: Refusing update from remote peer 8553047ecb914b2584d8cc730af9ad6a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260925 11:01:22.697762 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.697388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40440)
0925 11:01:22.697542 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.697611 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 11:01:22.697603 32724 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260925 11:01:22.697942 32156 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a -> Peer 1b678ee8f48245f7aaf9d8e30f224adb (127.24.159.254:36795): Couldn't send request to peer 1b678ee8f48245f7aaf9d8e30f224adb. Status: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260925 11:01:22.700652 32728 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8553047ecb914b2584d8cc730af9ad6a. Latest consensus state: current_term: 3 leader_uuid: "8553047ecb914b2584d8cc730af9ad6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:22.700917 32728 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:01:22.702272 32721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "8553047ecb914b2584d8cc730af9ad6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:22.702587 32721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:22.703066 32728 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "8553047ecb914b2584d8cc730af9ad6a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1b678ee8f48245f7aaf9d8e30f224adb" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36795 } } peers { permanent_uuid: "ddf176fd1cc24080b1c99912aa360f1e" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43627 } attrs { promote: false } } peers { permanent_uuid: "8553047ecb914b2584d8cc730af9ad6a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 41255 } attrs { promote: false } } }
I20260925 11:01:22.703292 32728 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddf176fd1cc24080b1c99912aa360f1e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:01:22.719100 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.712151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41826 (local address 127.24.159.194:34645)
0925 11:01:22.712614 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:22.712620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:22.712640 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:01:22.712712 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:22.712717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:22.712792 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:22.712886 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:22.713841 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.714830 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.716714 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.717948 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.718084 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.718164 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:22.718613 (+ 449us) server_negotiation.cc:299] Negotiation successful
0925 11:01:22.718752 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":808,"threads_started":1}
I20260925 11:01:22.721590 32735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.711507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:34645 (local address 127.0.0.1:41826)
0925 11:01:22.712101 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.712118 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:22.712221 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:22.713065 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:22.713069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:22.713283 (+ 214us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:22.713632 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.713645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.714984 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.714989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.716552 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.716564 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.718096 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.718102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.718315 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.718354 (+ 39us) client_negotiation.cc:769] Sending connection context
0925 11:01:22.718445 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 11:01:22.721179 (+ 2734us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":2630,"thread_start_us":322,"threads_started":1}
I20260925 11:01:22.722678 32429 tablet_service.cc:1467] Processing CreateTablet for tablet 0e14e8df737e45869e74ab3859d484ed (DEFAULT_TABLE table=test-table [id=987c20c3257b4e0e81acb9bfbf63f44c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:22.722957 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.712496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:43259 (local address 127.0.0.1:40142)
0925 11:01:22.712974 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.712989 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:22.713083 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:22.714954 (+ 1871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:22.714959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:22.714986 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:22.715398 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.715409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.716909 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.716926 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.719948 (+ 3022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.719959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.722164 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.722170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.722363 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.722396 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 11:01:22.722487 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 11:01:22.722605 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":5,"thread_start_us":540,"threads_started":1}
I20260925 11:01:22.723057 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.712849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.24.159.195:43259)
0925 11:01:22.714625 (+ 1776us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:22.714632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:22.714652 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:01:22.714722 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:22.714728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:22.714803 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:22.714911 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:22.715584 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.716739 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.720111 (+ 3372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.720893 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.721041 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.721111 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:22.722551 (+ 1440us) server_negotiation.cc:299] Negotiation successful
0925 11:01:22.722719 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1639,"thread_start_us":1516,"threads_started":1}
I20260925 11:01:22.723187 32429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e14e8df737e45869e74ab3859d484ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:22.728826 32570 tablet_service.cc:1467] Processing CreateTablet for tablet 0e14e8df737e45869e74ab3859d484ed (DEFAULT_TABLE table=test-table [id=987c20c3257b4e0e81acb9bfbf63f44c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:22.729286 32570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e14e8df737e45869e74ab3859d484ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:22.729470 32739 tablet_bootstrap.cc:492] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: Bootstrap starting.
I20260925 11:01:22.731281 32739 tablet_bootstrap.cc:654] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:22.731987 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.711131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35342)
0925 11:01:22.711277 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.711297 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:22.711422 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:22.726909 (+ 15487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:22.726915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:22.726941 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:22.727178 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.727187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.728444 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.728450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.730018 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.730032 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.731252 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.731259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.731433 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.731468 (+ 35us) client_negotiation.cc:769] Sending connection context
0925 11:01:22.731550 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 11:01:22.731680 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260925 11:01:22.732031 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.711245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35342 (local address 127.24.159.193:43673)
0925 11:01:22.726616 (+ 15371us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:22.726623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:22.726644 (+ 21us) server_negotiation.cc:407] Connection header received
0925 11:01:22.726710 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:22.726715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:22.726787 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:22.726891 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:22.727319 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.728298 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.730247 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.731047 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.731211 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.731296 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:22.731598 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 11:01:22.731732 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15237,"thread_start_us":15112,"threads_started":1}
I20260925 11:01:22.735073 32285 tablet_service.cc:1467] Processing CreateTablet for tablet 0e14e8df737e45869e74ab3859d484ed (DEFAULT_TABLE table=test-table [id=987c20c3257b4e0e81acb9bfbf63f44c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:22.735498 32285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e14e8df737e45869e74ab3859d484ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:22.735553 32739 tablet_bootstrap.cc:492] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: No bootstrap required, opened a new log
I20260925 11:01:22.735684 32739 ts_tablet_manager.cc:1397] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260925 11:01:22.735805 32741 tablet_bootstrap.cc:492] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: Bootstrap starting.
I20260925 11:01:22.736364 32739 raft_consensus.cc:348] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } }
I20260925 11:01:22.736598 32739 raft_consensus.cc:374] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:22.736678 32739 raft_consensus.cc:729] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1275fb03f9e748f281bc34fcab9cc133, State: Initialized, Role: FOLLOWER
I20260925 11:01:22.736908 32739 consensus_queue.cc:260] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } }
I20260925 11:01:22.737453 32741 tablet_bootstrap.cc:654] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:22.737664 32739 ts_tablet_manager.cc:1428] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 11:01:22.738847 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.738387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:48531)
0925 11:01:22.738615 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.738701 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260925 11:01:22.740990 32494 heartbeater.cc:503] Master 127.24.159.252:41255 was elected leader, sending a full tablet report...
I20260925 11:01:22.741765 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.740904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:57743)
0925 11:01:22.741530 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.741612 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525}
I20260925 11:01:22.742063 32744 tablet_bootstrap.cc:492] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd: Bootstrap starting.
I20260925 11:01:22.743641 32744 tablet_bootstrap.cc:654] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:22.745406 32744 tablet_bootstrap.cc:492] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd: No bootstrap required, opened a new log
I20260925 11:01:22.745536 32744 ts_tablet_manager.cc:1397] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 11:01:22.746147 32744 raft_consensus.cc:348] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } }
I20260925 11:01:22.746480 32744 raft_consensus.cc:374] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:22.746594 32744 raft_consensus.cc:729] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c8452f139574a7985ec962338123efd, State: Initialized, Role: FOLLOWER
I20260925 11:01:22.746747 32741 tablet_bootstrap.cc:492] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: No bootstrap required, opened a new log
I20260925 11:01:22.746886 32741 ts_tablet_manager.cc:1397] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260925 11:01:22.746861 32744 consensus_queue.cc:260] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } }
I20260925 11:01:22.747627 32741 raft_consensus.cc:348] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } }
I20260925 11:01:22.747947 32741 raft_consensus.cc:374] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:22.748083 32741 raft_consensus.cc:729] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f288844d41a440d8db0965a2ed7e6d7, State: Initialized, Role: FOLLOWER
I20260925 11:01:22.748334 32741 consensus_queue.cc:260] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } }
I20260925 11:01:22.748720 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.748005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.193:38517)
0925 11:01:22.748433 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.748533 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317}
I20260925 11:01:22.750123 32744 ts_tablet_manager.cc:1428] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260925 11:01:22.750379 32350 heartbeater.cc:503] Master 127.24.159.252:41255 was elected leader, sending a full tablet report...
I20260925 11:01:22.751096 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.750648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.193:33057)
0925 11:01:22.750860 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.750945 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260925 11:01:22.755738 32741 ts_tablet_manager.cc:1428] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260925 11:01:22.756243 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.755737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.195:33077)
0925 11:01:22.755998 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.756089 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260925 11:01:22.756223 32742 raft_consensus.cc:482] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:22.756479 32742 raft_consensus.cc:504] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } }
I20260925 11:01:22.757318 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.756855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.195:47167)
0925 11:01:22.757050 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.757139 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260925 11:01:22.757887 32635 heartbeater.cc:503] Master 127.24.159.252:41255 was elected leader, sending a full tablet report...
I20260925 11:01:22.758426 32742 leader_election.cc:288] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c8452f139574a7985ec962338123efd (127.24.159.193:43673), 9f288844d41a440d8db0965a2ed7e6d7 (127.24.159.195:43259)
I20260925 11:01:22.764567 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.759165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:35651 (local address 127.24.159.193:43673)
0925 11:01:22.759587 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:22.759593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:22.759610 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:01:22.759672 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:22.759677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:22.759746 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:22.760281 (+ 535us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:22.760425 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.761144 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.762949 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.763676 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.763839 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.763921 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:22.764192 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 11:01:22.764318 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260925 11:01:22.764629 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.758550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.24.159.194:35651)
0925 11:01:22.759354 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.759372 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:22.759493 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:22.759894 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:22.759899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:22.759926 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:22.760161 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.760170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.761263 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.761269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.762805 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.762817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.763864 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.763871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.764047 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.764075 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 11:01:22.764157 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 11:01:22.764309 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":62}
I20260925 11:01:22.764627 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.758902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:41507 (local address 127.24.159.195:43259)
0925 11:01:22.759080 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:22.759086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:22.759527 (+ 441us) server_negotiation.cc:407] Connection header received
0925 11:01:22.759880 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:22.759884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:22.759947 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:22.760005 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:22.760637 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.761292 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.763028 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.763789 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.763928 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.763999 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:22.764292 (+ 293us) server_negotiation.cc:299] Negotiation successful
0925 11:01:22.764414 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 11:01:22.765075 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.758852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:43259 (local address 127.24.159.194:41507)
0925 11:01:22.759434 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.759452 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:22.759732 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:22.760189 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:22.760193 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:22.760237 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:22.760498 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.760506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.761422 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.761426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.762886 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:22.762898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.764000 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.764006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:22.764150 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.764171 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 11:01:22.764256 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 11:01:22.764378 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":776,"threads_started":1}
I20260925 11:01:22.765331 32305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e14e8df737e45869e74ab3859d484ed" candidate_uuid: "1275fb03f9e748f281bc34fcab9cc133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8452f139574a7985ec962338123efd" is_pre_election: true
I20260925 11:01:22.765553 32305 raft_consensus.cc:2454] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1275fb03f9e748f281bc34fcab9cc133 in term 0.
I20260925 11:01:22.765825 32590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e14e8df737e45869e74ab3859d484ed" candidate_uuid: "1275fb03f9e748f281bc34fcab9cc133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f288844d41a440d8db0965a2ed7e6d7" is_pre_election: true
I20260925 11:01:22.766016 32382 leader_election.cc:302] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1275fb03f9e748f281bc34fcab9cc133, 6c8452f139574a7985ec962338123efd; no voters:
I20260925 11:01:22.766233 32590 raft_consensus.cc:2454] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1275fb03f9e748f281bc34fcab9cc133 in term 0.
I20260925 11:01:22.766578 32742 raft_consensus.cc:2781] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:22.766711 32742 raft_consensus.cc:482] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:22.766810 32742 raft_consensus.cc:3037] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:22.768925 32742 raft_consensus.cc:504] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } }
I20260925 11:01:22.769501 32742 leader_election.cc:288] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [CANDIDATE]: Term 1 election: Requested vote from peers 6c8452f139574a7985ec962338123efd (127.24.159.193:43673), 9f288844d41a440d8db0965a2ed7e6d7 (127.24.159.195:43259)
I20260925 11:01:22.769990 32590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e14e8df737e45869e74ab3859d484ed" candidate_uuid: "1275fb03f9e748f281bc34fcab9cc133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f288844d41a440d8db0965a2ed7e6d7"
I20260925 11:01:22.770073 32305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e14e8df737e45869e74ab3859d484ed" candidate_uuid: "1275fb03f9e748f281bc34fcab9cc133" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c8452f139574a7985ec962338123efd"
I20260925 11:01:22.770184 32590 raft_consensus.cc:3037] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:22.770295 32305 raft_consensus.cc:3037] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:22.772228 32305 raft_consensus.cc:2454] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1275fb03f9e748f281bc34fcab9cc133 in term 1.
I20260925 11:01:22.772508 32590 raft_consensus.cc:2454] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1275fb03f9e748f281bc34fcab9cc133 in term 1.
I20260925 11:01:22.772642 32382 leader_election.cc:302] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1275fb03f9e748f281bc34fcab9cc133, 6c8452f139574a7985ec962338123efd; no voters:
I20260925 11:01:22.772938 32742 raft_consensus.cc:2781] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:22.773396 32742 raft_consensus.cc:686] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 1 LEADER]: Becoming Leader. State: Replica: 1275fb03f9e748f281bc34fcab9cc133, State: Running, Role: LEADER
I20260925 11:01:22.773787 32742 consensus_queue.cc:237] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } }
I20260925 11:01:22.774552 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.774082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:39673)
0925 11:01:22.774264 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.774403 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 11:01:22.777145 32171 catalog_manager.cc:5495] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1275fb03f9e748f281bc34fcab9cc133 (127.24.159.194). New cstate: current_term: 1 leader_uuid: "1275fb03f9e748f281bc34fcab9cc133" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1275fb03f9e748f281bc34fcab9cc133" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 34645 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 } health_report { overall_health: UNKNOWN } } }
I20260925 11:01:22.800499 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.795398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41834 (local address 127.24.159.194:34645)
0925 11:01:22.795584 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:22.795590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:22.795608 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:01:22.795791 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:22.795795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:22.795883 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:22.796034 (+ 151us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:22.796377 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.797049 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.798327 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.798782 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.798886 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.799277 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:22.799885 (+ 608us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:22.800014 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:22.800138 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 11:01:22.800209 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 11:01:22.804618 32305 raft_consensus.cc:1261] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [term 1 FOLLOWER]: Refusing update from remote peer 1275fb03f9e748f281bc34fcab9cc133: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:22.804921 32590 raft_consensus.cc:1261] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [term 1 FOLLOWER]: Refusing update from remote peer 1275fb03f9e748f281bc34fcab9cc133: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:22.805370 32749 consensus_queue.cc:1035] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c8452f139574a7985ec962338123efd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:22.805783 32742 consensus_queue.cc:1035] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f288844d41a440d8db0965a2ed7e6d7" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 43259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:22.807858 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.807374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.193:57447)
0925 11:01:22.807597 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.807690 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260925 11:01:22.808116 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.807727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.195:49557)
0925 11:01:22.807848 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.807949 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260925 11:01:22.811292 32752 mvcc.cc:205] Tried to move back new op lower bound from 7333208403161145344 to 7333208403045797888. Current Snapshot: MvccSnapshot[applied={T|T < 7333208403161145344}]
I20260925 11:01:22.813475 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.812191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:51901)
0925 11:01:22.813218 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.813313 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":917}
I20260925 11:01:22.816172 32720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.815703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.195:48399)
0925 11:01:22.815920 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.816007 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260925 11:01:22.820304 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.819792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.194:37179)
0925 11:01:22.819986 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.820089 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 11:01:22.824466 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.821377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.24.159.193:35027)
0925 11:01:22.824077 (+ 2700us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:22.824190 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260925 11:01:22.835868 32172 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:22.836143 32172 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57656:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:22.841518 32172 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8553047ecb914b2584d8cc730af9ad6a: Sending DeleteTablet for 3 replicas of tablet 0e14e8df737e45869e74ab3859d484ed
I20260925 11:01:22.842696 32570 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e14e8df737e45869e74ab3859d484ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:22 UTC) from {username='oryx', principal='oryx/127.24.159.252@KRBTEST.COM'} at 127.0.0.1:40142
I20260925 11:01:22.842738 32285 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e14e8df737e45869e74ab3859d484ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:22 UTC) from {username='oryx', principal='oryx/127.24.159.252@KRBTEST.COM'} at 127.0.0.1:35342
I20260925 11:01:22.843138 32429 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e14e8df737e45869e74ab3859d484ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:22 UTC) from {username='oryx', principal='oryx/127.24.159.252@KRBTEST.COM'} at 127.0.0.1:41826
I20260925 11:01:22.844000 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 32138
I20260925 11:01:22.885028 32764 tablet_replica.cc:330] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: stopping tablet replica
I20260925 11:01:22.885294 32764 raft_consensus.cc:2229] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:01:22.885968 32764 raft_consensus.cc:2258] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:22.886593 32763 tablet_replica.cc:330] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: stopping tablet replica
I20260925 11:01:22.887409 32764 ts_tablet_manager.cc:1905] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:22.890398 32763 raft_consensus.cc:2229] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:22.890625 32762 tablet_replica.cc:330] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd: stopping tablet replica
I20260925 11:01:22.890796 32762 raft_consensus.cc:2229] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:22.890863 32763 raft_consensus.cc:2258] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:22.891072 32762 raft_consensus.cc:2258] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:22.892144 32763 ts_tablet_manager.cc:1905] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:22.898504 32762 ts_tablet_manager.cc:1905] T 0e14e8df737e45869e74ab3859d484ed P 6c8452f139574a7985ec962338123efd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:22.899794 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:22.845476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35358 (local address 127.24.159.193:43673)
0925 11:01:22.845647 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:22.845654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:22.882856 (+ 37202us) server_negotiation.cc:407] Connection header received
0925 11:01:22.883041 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:22.883048 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:22.883151 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:22.883273 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:22.885253 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.886465 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.888284 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:22.888901 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:22.889031 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:22.892244 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:22.892263 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:22.892285 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:22.892326 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:22.896205 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:22.897121 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:22.897132 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:22.897136 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:22.897291 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:22.897795 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:22.897801 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:22.897804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:22.898333 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:22.898607 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:22.899179 (+ 572us) server_negotiation.cc:299] Negotiation successful
0925 11:01:22.899416 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:01:22.900065 32763 ts_tablet_manager.cc:1918] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:22.900298 32763 log.cc:1196] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/0e14e8df737e45869e74ab3859d484ed
I20260925 11:01:22.901569 32345 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:22.903702 32764 ts_tablet_manager.cc:1918] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:22.903955 32764 log.cc:1196] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/0e14e8df737e45869e74ab3859d484ed
I20260925 11:01:22.904426 32763 ts_tablet_manager.cc:1939] T 0e14e8df737e45869e74ab3859d484ed P 9f288844d41a440d8db0965a2ed7e6d7: Deleting consensus metadata
I20260925 11:01:22.904505 32764 ts_tablet_manager.cc:1939] T 0e14e8df737e45869e74ab3859d484ed P 1275fb03f9e748f281bc34fcab9cc133: Deleting consensus metadata
I20260925 11:01:22.905704 32157 catalog_manager.cc:4841] TS 9f288844d41a440d8db0965a2ed7e6d7 (127.24.159.195:43259): tablet 0e14e8df737e45869e74ab3859d484ed (table test-table [id=987c20c3257b4e0e81acb9bfbf63f44c]) successfully deleted
I20260925 11:01:22.905709 32156 catalog_manager.cc:4841] TS 1275fb03f9e748f281bc34fcab9cc133 (127.24.159.194:34645): tablet 0e14e8df737e45869e74ab3859d484ed (table test-table [id=987c20c3257b4e0e81acb9bfbf63f44c]) successfully deleted
I20260925 11:01:22.999408 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 32138
I20260925 11:01:23.014604 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 32361
I20260925 11:01:23.015029 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.014556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35360)
0925 11:01:23.014748 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.014856 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260925 11:01:23.015491 32156 catalog_manager.cc:4584] TS 6c8452f139574a7985ec962338123efd (127.24.159.193:43673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e14e8df737e45869e74ab3859d484ed: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
I20260925 11:01:23.029950 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.015440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41840 (local address 127.24.159.194:34645)
0925 11:01:23.015617 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:23.015623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:23.015795 (+ 172us) server_negotiation.cc:407] Connection header received
0925 11:01:23.016052 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:23.016058 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:23.016155 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:23.016251 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:23.017869 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:23.018703 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:23.020446 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:23.020887 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:23.020984 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:23.024035 (+ 3051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:23.024048 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:23.024064 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:23.024100 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:23.027171 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:23.027915 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:23.027925 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:23.027927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:23.028078 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:23.028446 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:23.028450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:23.028452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:23.028812 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:23.029127 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:23.029397 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 11:01:23.029603 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:01:23.030603 32489 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:23.069043 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.068558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35372)
0925 11:01:23.068783 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.068882 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260925 11:01:23.070039 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.069722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35388)
0925 11:01:23.069859 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.069933 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 11:01:23.135696 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.135265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40456)
0925 11:01:23.135451 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.135542 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 11:01:23.136619 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.136336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36795 (local address 127.0.0.1:40462)
0925 11:01:23.136454 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.136519 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260925 11:01:23.145824 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.145365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35396)
0925 11:01:23.145569 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.145658 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 11:01:23.147423 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.147058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35410)
0925 11:01:23.147199 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.147277 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260925 11:01:23.147413 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 32361
I20260925 11:01:23.162459 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 32507
I20260925 11:01:23.175251 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.163150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40154 (local address 127.24.159.195:43259)
0925 11:01:23.163313 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:23.163319 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:23.163339 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:01:23.163513 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:23.163518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:23.163579 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:23.163660 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:23.164935 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:23.165649 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:23.166971 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:23.167344 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:23.167461 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:23.169834 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:23.169849 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:23.169865 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:23.169900 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:23.172664 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:23.173309 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:23.173316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:23.173318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:23.173449 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:23.173807 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:23.173811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:23.173813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:23.174183 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:23.174570 (+ 387us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:23.174775 (+ 205us) server_negotiation.cc:299] Negotiation successful
0925 11:01:23.174941 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 11:01:23.175848 32630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:23.263270 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.262811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35418)
0925 11:01:23.263034 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.263122 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260925 11:01:23.264205 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.263878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35434)
0925 11:01:23.264025 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.264093 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 11:01:23.291546 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 32507
I20260925 11:01:23.309435 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 31846
I20260925 11:01:23.310473 32013 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:23.414538 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.413976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35444)
0925 11:01:23.414244 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.414361 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260925 11:01:23.415534 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:23.415156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43673 (local address 127.0.0.1:35450)
0925 11:01:23.415306 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:23.415402 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:01:23.424220 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 31846
I20260925 11:01:23.441075 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 31945
I20260925 11:01:23.442073 32202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:23.559593 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 31945
2026-09-25T11:01:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10410 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:01:23 dist-test-slave-trms krb5kdc[305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:01:23 dist-test-slave-trms krb5kdc[305](info): set up 2 sockets
Sep 25 11:01:23 dist-test-slave-trms krb5kdc[305](info): commencing operation
krb5kdc: starting...
W20260925 11:01:24.559017 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:01:24 dist-test-slave-trms krb5kdc[305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:01:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:24Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.254.keytab.
Entry for principal oryx/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.254.keytab.
I20260925 11:01:24.760149 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36441
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.254.keytab
--principal=oryx/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:36441
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:25.067888 318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:25.068162 318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:25.068289 318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:25.075387 318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:25.075503 318 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260925 11:01:25.075570 318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:25.075659 318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:25.075721 318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:25.075800 318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:25.085491 318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:36441
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.254.keytab
--principal=oryx/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:25.086805 318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:25.088116 318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:25.099203 324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:25.099233 326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:25.099841 323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:25.100318 318 server_base.cc:1034] running on GCE node
I20260925 11:01:25.101006 318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:25.102339 318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:25.103545 318 hybrid_clock.cc:648] HybridClock initialized: now 1790334085103487 us; error 39 us; skew 500 ppm
I20260925 11:01:25.103852 318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:25 dist-test-slave-trms krb5kdc[305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334085, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:25.106701 318 init.cc:377] Logged in from keytab as oryx/127.24.159.254@KRBTEST.COM (short username oryx)
I20260925 11:01:25.108062 318 webserver.cc:460] Webserver started at http://127.24.159.254:42001/ using document root <none> and password file <none>
I20260925 11:01:25.108734 318 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:25.108822 318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:25.109124 318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:25.112152 318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "6524dbf9b7274cafbfe0442612b32e68"
format_stamp: "Formatted at 2026-09-25 11:01:25 on dist-test-slave-trms"
server_key: "7ddaadb82f090576d80831023a614784"
server_key_iv: "7496c4eecce73dcb3e27e71a5d25eb99"
server_key_version: "encryptionkey@0"
I20260925 11:01:25.112898 318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "6524dbf9b7274cafbfe0442612b32e68"
format_stamp: "Formatted at 2026-09-25 11:01:25 on dist-test-slave-trms"
server_key: "7ddaadb82f090576d80831023a614784"
server_key_iv: "7496c4eecce73dcb3e27e71a5d25eb99"
server_key_version: "encryptionkey@0"
I20260925 11:01:25.117825 318 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 11:01:25.121066 333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:25.122355 318 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:25.122514 318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "6524dbf9b7274cafbfe0442612b32e68"
format_stamp: "Formatted at 2026-09-25 11:01:25 on dist-test-slave-trms"
server_key: "7ddaadb82f090576d80831023a614784"
server_key_iv: "7496c4eecce73dcb3e27e71a5d25eb99"
server_key_version: "encryptionkey@0"
I20260925 11:01:25.122695 318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:25.137233 318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:25.141484 318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:25.141726 318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:25.161231 318 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:36441
I20260925 11:01:25.161248 394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:36441 every 8 connection(s)
I20260925 11:01:25.162644 318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:25.166724 395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:25.168002 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 318
I20260925 11:01:25.168171 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:25.168781 25215 external_mini_cluster.cc:1428] Setting key 57f0879205232f5cf2221b28104b6dae
I20260925 11:01:25.180415 395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68: Bootstrap starting.
Sep 25 11:01:25 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334084, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:25.184103 395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:25.185395 395 log.cc:824] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:25.192142 395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68: No bootstrap required, opened a new log
I20260925 11:01:25.192440 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:25.170652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51754 (local address 127.24.159.254:36441)
0925 11:01:25.171463 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:25.171478 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:25.171530 (+ 52us) server_negotiation.cc:407] Connection header received
0925 11:01:25.173128 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:25.173160 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:25.173675 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:25.174102 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:25.175340 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.177246 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.178455 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.179449 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.179630 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:25.183619 (+ 3989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:25.183629 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:25.183647 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:25.183696 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:25.187504 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:25.188348 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:25.188357 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:25.188367 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:25.188582 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:25.189166 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:25.189175 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:25.189178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:25.189914 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:25.190225 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:25.190695 (+ 470us) server_negotiation.cc:299] Negotiation successful
0925 11:01:25.191129 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":307,"threads_started":1}
I20260925 11:01:25.203308 395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6524dbf9b7274cafbfe0442612b32e68" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36441 } }
I20260925 11:01:25.203773 395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:25.203879 395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6524dbf9b7274cafbfe0442612b32e68, State: Initialized, Role: FOLLOWER
I20260925 11:01:25.204538 395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6524dbf9b7274cafbfe0442612b32e68" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36441 } }
I20260925 11:01:25.204802 395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:25.204943 395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:25.205111 395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:25.208103 395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6524dbf9b7274cafbfe0442612b32e68" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36441 } }
I20260925 11:01:25.208678 395 leader_election.cc:302] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6524dbf9b7274cafbfe0442612b32e68; no voters:
I20260925 11:01:25.209318 395 leader_election.cc:288] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:25.209448 400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:25.210091 400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [term 1 LEADER]: Becoming Leader. State: Replica: 6524dbf9b7274cafbfe0442612b32e68, State: Running, Role: LEADER
I20260925 11:01:25.210707 400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6524dbf9b7274cafbfe0442612b32e68" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36441 } }
I20260925 11:01:25.211609 395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:25.213726 402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6524dbf9b7274cafbfe0442612b32e68. Latest consensus state: current_term: 1 leader_uuid: "6524dbf9b7274cafbfe0442612b32e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6524dbf9b7274cafbfe0442612b32e68" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36441 } } }
I20260925 11:01:25.213356 401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6524dbf9b7274cafbfe0442612b32e68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6524dbf9b7274cafbfe0442612b32e68" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36441 } } }
I20260925 11:01:25.218508 401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:25.219367 402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:25.219364 409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:25.225229 409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:25.243402 409 catalog_manager.cc:1347] Generated new cluster ID: 799e977a03d64a639bd58f4027f42da8
I20260925 11:01:25.243530 409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:25.276744 409 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:25.278715 409 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:25.290115 409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68: Generated new TSK 0
I20260925 11:01:25.291226 409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab.
Entry for principal oryx/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab.
I20260925 11:01:25.341125 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36441
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab
--principal=oryx/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:25.657215 421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:25.657506 421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:25.657641 421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:25.664716 421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:25.664834 421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:25.664954 421 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 11:01:25.665060 421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:25.674628 421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab
--principal=oryx/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.159.254:36441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:25.675928 421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:25.677245 421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:25.689870 426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:25.689843 429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:25.689955 427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:25.690552 421 server_base.cc:1034] running on GCE node
I20260925 11:01:25.691201 421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:25.692119 421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:25.693326 421 hybrid_clock.cc:648] HybridClock initialized: now 1790334085693264 us; error 38 us; skew 500 ppm
I20260925 11:01:25.693611 421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:25 dist-test-slave-trms krb5kdc[305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334085, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:25.696683 421 init.cc:377] Logged in from keytab as oryx/127.24.159.193@KRBTEST.COM (short username oryx)
I20260925 11:01:25.697955 421 webserver.cc:460] Webserver started at http://127.24.159.193:44377/ using document root <none> and password file <none>
I20260925 11:01:25.698761 421 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:25.698881 421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:25.699227 421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:25.702081 421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "2c2f37558b864d6a85b3fce37215eb60"
format_stamp: "Formatted at 2026-09-25 11:01:25 on dist-test-slave-trms"
server_key: "ac81f02a2a4d1f546b5f876534bebd63"
server_key_iv: "357a09c34d9199b51a99ab6467a3f4f1"
server_key_version: "encryptionkey@0"
I20260925 11:01:25.702826 421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "2c2f37558b864d6a85b3fce37215eb60"
format_stamp: "Formatted at 2026-09-25 11:01:25 on dist-test-slave-trms"
server_key: "ac81f02a2a4d1f546b5f876534bebd63"
server_key_iv: "357a09c34d9199b51a99ab6467a3f4f1"
server_key_version: "encryptionkey@0"
I20260925 11:01:25.707544 421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260925 11:01:25.710728 436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:25.712001 421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 11:01:25.712121 421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "2c2f37558b864d6a85b3fce37215eb60"
format_stamp: "Formatted at 2026-09-25 11:01:25 on dist-test-slave-trms"
server_key: "ac81f02a2a4d1f546b5f876534bebd63"
server_key_iv: "357a09c34d9199b51a99ab6467a3f4f1"
server_key_version: "encryptionkey@0"
I20260925 11:01:25.712285 421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:25.728684 421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:25.732707 421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:25.732921 421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:25.735440 421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:25.735519 421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:25.735621 421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:25.735683 421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 11:01:25 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334085, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:25.757757 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:25.735859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:38955 (local address 127.24.159.254:36441)
0925 11:01:25.736580 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:25.736587 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:25.737571 (+ 984us) server_negotiation.cc:407] Connection header received
0925 11:01:25.739745 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:25.739753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:25.739851 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:25.739975 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:25.742310 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.743269 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.745622 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.746336 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.746482 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:25.751039 (+ 4557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:25.751046 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:25.751051 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:25.751096 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:25.754064 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:25.754958 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:25.754963 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:25.754967 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:25.755096 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:25.755591 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:25.755596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:25.755600 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:25.755928 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:25.756085 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:25.757139 (+ 1054us) server_negotiation.cc:299] Negotiation successful
0925 11:01:25.757373 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":483,"threads_started":1}
I20260925 11:01:25.759599 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:25.736563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36441 (local address 127.24.159.193:38955)
0925 11:01:25.737382 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:25.737436 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:25.739334 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:25.740245 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:25.740253 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:25.740959 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:25.741846 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:25.741868 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.743428 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.743432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:25.745390 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:25.745407 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.746601 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.746609 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:25.746908 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:25.747712 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:25.747739 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:25.750804 (+ 3065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:25.754215 (+ 3411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:25.754228 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:25.754246 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:25.754832 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:25.755210 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:25.755215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:25.755218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:25.755426 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:25.756076 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:25.756086 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:25.756629 (+ 543us) client_negotiation.cc:769] Sending connection context
0925 11:01:25.757014 (+ 385us) client_negotiation.cc:240] Negotiation successful
0925 11:01:25.757686 (+ 672us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":336,"threads_started":1}
I20260925 11:01:25.777204 421 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:43785
I20260925 11:01:25.777221 556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:43785 every 8 connection(s)
I20260925 11:01:25.778416 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:25.765145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:42611 (local address 127.24.159.254:36441)
0925 11:01:25.765331 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:25.765338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:25.766258 (+ 920us) server_negotiation.cc:407] Connection header received
0925 11:01:25.766548 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:25.766553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:25.766637 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:25.766722 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:25.767921 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.768868 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.770158 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.770684 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.770804 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:25.772836 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:25.772841 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:25.772845 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:25.772875 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:25.775619 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:25.776422 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:25.776428 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:25.776432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:25.776596 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:25.777034 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:25.777039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:25.777043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:25.777364 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:25.777523 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:25.777857 (+ 334us) server_negotiation.cc:299] Negotiation successful
0925 11:01:25.778076 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:01:25.778431 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:25.765135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36441 (local address 127.24.159.193:42611)
0925 11:01:25.766145 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:25.766170 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:25.766341 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:25.766779 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:25.766785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:25.767370 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:25.767769 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:25.767784 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.769015 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.769020 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:25.770026 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:25.770037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:25.771045 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:25.771054 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:25.771253 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:25.771940 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:25.771976 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:25.772665 (+ 689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:25.775734 (+ 3069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:25.775738 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:25.775742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:25.776263 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:25.776721 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:25.776727 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:25.776731 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:25.776902 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:25.777473 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:25.777478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:25.777629 (+ 151us) client_negotiation.cc:769] Sending connection context
0925 11:01:25.777748 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 11:01:25.778054 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"thread_start_us":661,"threads_started":1}
I20260925 11:01:25.778841 421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:25.780794 557 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36441
I20260925 11:01:25.781349 557 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:25.782070 557 heartbeater.cc:511] Master 127.24.159.254:36441 requested a full tablet report, sending...
I20260925 11:01:25.782478 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 421
I20260925 11:01:25.782641 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:01:25.783335 25215 external_mini_cluster.cc:1428] Setting key 86abda000067357e4175ad4f1e949749
I20260925 11:01:25.783978 350 ts_manager.cc:194] Registered new tserver with Master: 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785)
I20260925 11:01:25.785535 350 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:38955
WARNING: no policy specified for oryx/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab.
Entry for principal oryx/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab.
I20260925 11:01:25.825481 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36441
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab
--principal=oryx/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:26.135149 561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:26.135409 561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:26.135550 561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:26.142710 561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:26.142818 561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:26.142961 561 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 11:01:26.143081 561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:26.152825 561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab
--principal=oryx/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.159.254:36441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:26.154052 561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:26.155337 561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:26.167977 566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:26.168146 567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:26.169553 561 server_base.cc:1034] running on GCE node
W20260925 11:01:26.170610 569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:26.171327 561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:26.172366 561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:26.173615 561 hybrid_clock.cc:648] HybridClock initialized: now 1790334086173530 us; error 50 us; skew 500 ppm
I20260925 11:01:26.173931 561 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:26 dist-test-slave-trms krb5kdc[305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334086, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:26.177438 561 init.cc:377] Logged in from keytab as oryx/127.24.159.194@KRBTEST.COM (short username oryx)
I20260925 11:01:26.178866 561 webserver.cc:460] Webserver started at http://127.24.159.194:46001/ using document root <none> and password file <none>
I20260925 11:01:26.179535 561 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:26.179625 561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:26.179908 561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:26.182718 561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "7456b6e5a70240dcb20f7a32ddbd79da"
format_stamp: "Formatted at 2026-09-25 11:01:26 on dist-test-slave-trms"
server_key: "db7de162cf0f6111500bc46768e686d8"
server_key_iv: "6a5d1b3e6b0123fb32e4585b2657f078"
server_key_version: "encryptionkey@0"
I20260925 11:01:26.183439 561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "7456b6e5a70240dcb20f7a32ddbd79da"
format_stamp: "Formatted at 2026-09-25 11:01:26 on dist-test-slave-trms"
server_key: "db7de162cf0f6111500bc46768e686d8"
server_key_iv: "6a5d1b3e6b0123fb32e4585b2657f078"
server_key_version: "encryptionkey@0"
I20260925 11:01:26.188181 561 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 11:01:26.191543 576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:26.192785 561 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:01:26.192934 561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "7456b6e5a70240dcb20f7a32ddbd79da"
format_stamp: "Formatted at 2026-09-25 11:01:26 on dist-test-slave-trms"
server_key: "db7de162cf0f6111500bc46768e686d8"
server_key_iv: "6a5d1b3e6b0123fb32e4585b2657f078"
server_key_version: "encryptionkey@0"
I20260925 11:01:26.193136 561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:26.210247 561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:26.214669 561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:26.214913 561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:26.217623 561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:26.217730 561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:26.217851 561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:26.217918 561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 11:01:26 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334086, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:26.242612 585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.219010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36441 (local address 127.24.159.194:43531)
0925 11:01:26.219833 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:26.219875 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:26.221863 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:26.222832 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:26.222848 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:26.223592 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:26.224583 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.224604 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.226074 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.226081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.228733 (+ 2652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.228761 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.229724 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.229733 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.230067 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.230873 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:26.230928 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:26.234068 (+ 3140us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:26.237517 (+ 3449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:26.237527 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:26.237545 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:26.238215 (+ 670us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:26.238692 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:26.238697 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:26.238701 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:26.239007 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:26.239638 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:26.239650 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:26.240182 (+ 532us) client_negotiation.cc:769] Sending connection context
0925 11:01:26.240559 (+ 377us) client_negotiation.cc:240] Negotiation successful
0925 11:01:26.241146 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":345,"threads_started":1}
I20260925 11:01:26.243438 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.218303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:43531 (local address 127.24.159.254:36441)
0925 11:01:26.218512 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:26.218518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:26.220003 (+ 1485us) server_negotiation.cc:407] Connection header received
0925 11:01:26.222303 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:26.222311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:26.222410 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:26.222531 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:26.224803 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.225917 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.228918 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.229460 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.229599 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.234317 (+ 4718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:26.234322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:26.234325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:26.234361 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:26.237382 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.238415 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.238420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.238423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.238581 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.239120 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.239125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.239129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.239504 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:26.239662 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:26.242993 (+ 3331us) server_negotiation.cc:299] Negotiation successful
0925 11:01:26.243161 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 11:01:26.257912 561 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:44161
I20260925 11:01:26.257923 695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:44161 every 8 connection(s)
I20260925 11:01:26.259287 561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:26.262084 696 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36441
I20260925 11:01:26.262346 696 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:26.263072 696 heartbeater.cc:511] Master 127.24.159.254:36441 requested a full tablet report, sending...
I20260925 11:01:26.263451 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 561
I20260925 11:01:26.263603 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:01:26.264070 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.249897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:33335 (local address 127.24.159.254:36441)
0925 11:01:26.250069 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:26.250075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:26.251000 (+ 925us) server_negotiation.cc:407] Connection header received
0925 11:01:26.251177 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:26.251183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:26.251260 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:26.251358 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:26.252543 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.253441 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.255518 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.256046 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.256146 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.258548 (+ 2402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:26.258554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:26.258557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:26.258591 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:26.261367 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.262045 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.262049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.262051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.262184 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.262605 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.262611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.262615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.262975 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:26.263104 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:26.263523 (+ 419us) server_negotiation.cc:299] Negotiation successful
0925 11:01:26.263734 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260925 11:01:26.264107 25215 external_mini_cluster.cc:1428] Setting key f157cb48e5254b3b7a21ee4d42ccacf2
I20260925 11:01:26.264161 678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.249970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36441 (local address 127.24.159.194:33335)
0925 11:01:26.250910 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:26.250932 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:26.251065 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:26.251419 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:26.251428 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:26.252003 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:26.252391 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.252403 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.253600 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.253606 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.255373 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.255395 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.256225 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.256231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.256397 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.257064 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:26.257093 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:26.258368 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:26.261510 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:26.261515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:26.261519 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:26.261944 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:26.262325 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:26.262331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:26.262339 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:26.262497 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:26.263090 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:26.263095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:26.263285 (+ 190us) client_negotiation.cc:769] Sending connection context
0925 11:01:26.263397 (+ 112us) client_negotiation.cc:240] Negotiation successful
0925 11:01:26.263737 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":350,"threads_started":1}
I20260925 11:01:26.264571 350 ts_manager.cc:194] Registered new tserver with Master: 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194:44161)
I20260925 11:01:26.265378 350 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:43531
WARNING: no policy specified for oryx/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab.
Entry for principal oryx/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab.
I20260925 11:01:26.306636 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36441
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab
--principal=oryx/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:26.617429 700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:26.617692 700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:26.617818 700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:26.624832 700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:26.624941 700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:26.625063 700 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260925 11:01:26.625169 700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:26.634554 700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab
--principal=oryx/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.159.254:36441
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:26.635712 700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:26.637002 700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:26.649241 708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:26.649262 706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:26.649453 705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:26.649830 700 server_base.cc:1034] running on GCE node
I20260925 11:01:26.650498 700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:26.651345 700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:26.652547 700 hybrid_clock.cc:648] HybridClock initialized: now 1790334086652481 us; error 41 us; skew 500 ppm
I20260925 11:01:26.652832 700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:26 dist-test-slave-trms krb5kdc[305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334086, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:26.655756 700 init.cc:377] Logged in from keytab as oryx/127.24.159.195@KRBTEST.COM (short username oryx)
I20260925 11:01:26.657076 700 webserver.cc:460] Webserver started at http://127.24.159.195:42357/ using document root <none> and password file <none>
I20260925 11:01:26.657724 700 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:26.657814 700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:26.658102 700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:26.660836 700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "cf778878acfe42ff82d92a410c4d082d"
format_stamp: "Formatted at 2026-09-25 11:01:26 on dist-test-slave-trms"
server_key: "6798c8ed574144db78e561dbbefc722c"
server_key_iv: "572ccc2aae4dab404b2a133af27f32b6"
server_key_version: "encryptionkey@0"
I20260925 11:01:26.661559 700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "cf778878acfe42ff82d92a410c4d082d"
format_stamp: "Formatted at 2026-09-25 11:01:26 on dist-test-slave-trms"
server_key: "6798c8ed574144db78e561dbbefc722c"
server_key_iv: "572ccc2aae4dab404b2a133af27f32b6"
server_key_version: "encryptionkey@0"
I20260925 11:01:26.666286 700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 11:01:26.669517 715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:26.670780 700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:26.670918 700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "cf778878acfe42ff82d92a410c4d082d"
format_stamp: "Formatted at 2026-09-25 11:01:26 on dist-test-slave-trms"
server_key: "6798c8ed574144db78e561dbbefc722c"
server_key_iv: "572ccc2aae4dab404b2a133af27f32b6"
server_key_version: "encryptionkey@0"
I20260925 11:01:26.671104 700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:26.697834 700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:26.701882 700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:26.702123 700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:26.704581 700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:26.704658 700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:26.704779 700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:26.704830 700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 25 11:01:26 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334086, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:26.725970 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.705242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:58311 (local address 127.24.159.254:36441)
0925 11:01:26.705427 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:26.705432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:26.706850 (+ 1418us) server_negotiation.cc:407] Connection header received
0925 11:01:26.708893 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:26.708900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:26.709005 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:26.709079 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:26.711603 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.712491 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.714120 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.714657 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.714791 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.719575 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:26.719583 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:26.719588 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:26.719635 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:26.722817 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.723528 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.723534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.723538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.723666 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.724079 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.724084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.724088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.724368 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:26.724500 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:26.725323 (+ 823us) server_negotiation.cc:299] Negotiation successful
0925 11:01:26.725596 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:01:26.726940 724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.705828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36441 (local address 127.24.159.195:58311)
0925 11:01:26.706657 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:26.706714 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:26.708553 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:26.709621 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:26.709632 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:26.710469 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:26.711401 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.711432 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.712646 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.712652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.713980 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.713992 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.714935 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.714943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.715266 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.716081 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:26.716115 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:26.719296 (+ 3181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:26.722971 (+ 3675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:26.722980 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:26.722997 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:26.723402 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:26.723770 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:26.723776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:26.723779 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:26.723963 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:26.724520 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:26.724529 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:26.724891 (+ 362us) client_negotiation.cc:769] Sending connection context
0925 11:01:26.725160 (+ 269us) client_negotiation.cc:240] Negotiation successful
0925 11:01:26.725674 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":345,"threads_started":1}
I20260925 11:01:26.746413 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.732285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:38027 (local address 127.24.159.254:36441)
0925 11:01:26.732467 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:26.732473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:26.733070 (+ 597us) server_negotiation.cc:407] Connection header received
0925 11:01:26.733275 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:26.733281 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:26.733369 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:26.733485 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:26.734705 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.735776 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.738079 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.738545 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.738658 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.740811 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:26.740818 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:26.740823 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:26.740862 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:26.743679 (+ 2817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.744562 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.744567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.744571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.744694 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.745034 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.745039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.745043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.745310 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:26.745437 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:26.745821 (+ 384us) server_negotiation.cc:299] Negotiation successful
0925 11:01:26.746052 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:01:26.746538 780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.732261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36441 (local address 127.24.159.195:38027)
0925 11:01:26.732978 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:26.732994 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:26.733125 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:26.733522 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:26.733527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:26.734083 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:26.734538 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.734551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.735995 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.736004 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.737925 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.737942 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.738753 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.738760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.738916 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.739556 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:26.739576 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:26.740628 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:26.743830 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:26.743836 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:26.743840 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:26.744452 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:26.744794 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:26.744800 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:26.744804 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:26.744937 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:26.745419 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:26.745423 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:26.745595 (+ 172us) client_negotiation.cc:769] Sending connection context
0925 11:01:26.745708 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 11:01:26.746044 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":425,"threads_started":1}
I20260925 11:01:26.753919 700 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:34953
I20260925 11:01:26.753942 834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:34953 every 8 connection(s)
I20260925 11:01:26.755097 700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:26.756872 835 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36441
I20260925 11:01:26.757071 835 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:26.757663 835 heartbeater.cc:511] Master 127.24.159.254:36441 requested a full tablet report, sending...
I20260925 11:01:26.758922 349 ts_manager.cc:194] Registered new tserver with Master: cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953)
I20260925 11:01:26.759720 349 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:58311
I20260925 11:01:26.763670 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 700
I20260925 11:01:26.763796 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:01:26.764179 25215 external_mini_cluster.cc:1428] Setting key 4db2e2c77d6b6ef152cf4bf194d65806
I20260925 11:01:26.766887 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:01:26.788616 557 heartbeater.cc:503] Master 127.24.159.254:36441 was elected leader, sending a full tablet report...
Sep 25 11:01:26 dist-test-slave-trms krb5kdc[305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334086, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 25 11:01:26 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:26 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 11:01:26.811970 445 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:26.802374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51758 (local address 127.24.159.254:36441)
0925 11:01:26.802527 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:26.802531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:26.802822 (+ 291us) server_negotiation.cc:407] Connection header received
0925 11:01:26.802975 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:26.802979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:26.803040 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:26.803129 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:26.804297 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.805011 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.806315 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.806693 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.806789 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.811671 (+ 4882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51758: BlockingRecv error: recv got EOF from 127.0.0.1:51758 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 11:01:26 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334086, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:26.827333 445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.815298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51762 (local address 127.24.159.254:36441)
0925 11:01:26.815504 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:26.815510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:26.815823 (+ 313us) server_negotiation.cc:407] Connection header received
0925 11:01:26.815978 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:26.815983 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:26.816050 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:26.816130 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:26.817305 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.817996 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.819213 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.819620 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.819764 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.822995 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:26.823000 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:26.823003 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:26.823033 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:26.825133 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.825735 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.825739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.825741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.825853 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:26.826156 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:26.826159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:26.826161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:26.826468 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:26.826749 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:26.826915 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 11:01:26.827096 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:01:26.830915 349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:01:26.837572 349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:26.859543 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.849896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35690 (local address 127.24.159.193:43785)
0925 11:01:26.851966 (+ 2070us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:26.851975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:26.852562 (+ 587us) server_negotiation.cc:407] Connection header received
0925 11:01:26.852659 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:26.852665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:26.852915 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:26.853119 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:26.853700 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.854987 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.857358 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.858411 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.858599 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.858790 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:26.859060 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 11:01:26.859224 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1922,"thread_start_us":1804,"threads_started":1}
I20260925 11:01:26.859596 853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.849309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:34953 (local address 127.0.0.1:38902)
0925 11:01:26.850041 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:26.850085 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:26.850288 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:26.851382 (+ 1094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:26.851399 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:26.851435 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:26.851859 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.851871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.854593 (+ 2722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.854598 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.857092 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.857110 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.858902 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.858908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.859066 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.859095 (+ 29us) client_negotiation.cc:769] Sending connection context
0925 11:01:26.859147 (+ 52us) client_negotiation.cc:240] Negotiation successful
0925 11:01:26.859243 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":280,"threads_started":1}
I20260925 11:01:26.859619 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.849389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43785 (local address 127.0.0.1:35690)
0925 11:01:26.852286 (+ 2897us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:26.852304 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:26.852425 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:26.853096 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:26.853100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:26.853121 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:26.853518 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.853530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.855152 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.855157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.857242 (+ 2085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.857253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.858616 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.858622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.858808 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.858872 (+ 64us) client_negotiation.cc:769] Sending connection context
0925 11:01:26.859005 (+ 133us) client_negotiation.cc:240] Negotiation successful
0925 11:01:26.859181 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2777,"mutex_wait_us":4,"thread_start_us":2627,"threads_started":1}
I20260925 11:01:26.860695 857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.852348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:44161 (local address 127.0.0.1:56258)
0925 11:01:26.852828 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:26.852843 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:26.852948 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:26.854457 (+ 1509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:26.854461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:26.854487 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:26.854892 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.854903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.856366 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.856372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.858175 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:26.858222 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.860007 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.860014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:26.860170 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.860202 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 11:01:26.860281 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 11:01:26.860396 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":254,"threads_started":1}
I20260925 11:01:26.860898 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.849547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38902 (local address 127.24.159.195:34953)
0925 11:01:26.850689 (+ 1142us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:26.850698 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:26.850736 (+ 38us) server_negotiation.cc:407] Connection header received
0925 11:01:26.850840 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:26.850848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:26.851140 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:26.851342 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:26.852104 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.854425 (+ 2321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.857292 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.858691 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.860185 (+ 1494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.860354 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:26.860432 (+ 78us) server_negotiation.cc:299] Negotiation successful
0925 11:01:26.860591 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":970,"thread_start_us":797,"threads_started":1}
I20260925 11:01:26.861609 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:26.852575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56258 (local address 127.24.159.194:44161)
0925 11:01:26.853886 (+ 1311us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:26.853906 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:26.853931 (+ 25us) server_negotiation.cc:407] Connection header received
0925 11:01:26.854018 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:26.854025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:26.854271 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:26.854447 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:26.855086 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.856211 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.858358 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:26.859562 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:26.860929 (+ 1367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:26.861125 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:26.861202 (+ 77us) server_negotiation.cc:299] Negotiation successful
0925 11:01:26.861351 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1158,"thread_start_us":1043,"threads_started":1}
I20260925 11:01:26.864265 487 tablet_service.cc:1467] Processing CreateTablet for tablet c1b99ef5cf8e41e6a1f130972e61dcb0 (DEFAULT_TABLE table=test-table [id=bb0ae8d3ba704abc97a893c6eaa39f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:26.864459 626 tablet_service.cc:1467] Processing CreateTablet for tablet c1b99ef5cf8e41e6a1f130972e61dcb0 (DEFAULT_TABLE table=test-table [id=bb0ae8d3ba704abc97a893c6eaa39f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:26.864456 765 tablet_service.cc:1467] Processing CreateTablet for tablet c1b99ef5cf8e41e6a1f130972e61dcb0 (DEFAULT_TABLE table=test-table [id=bb0ae8d3ba704abc97a893c6eaa39f4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:26.865587 487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1b99ef5cf8e41e6a1f130972e61dcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:26.865705 626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1b99ef5cf8e41e6a1f130972e61dcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:26.865705 765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1b99ef5cf8e41e6a1f130972e61dcb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:26.876890 859 tablet_bootstrap.cc:492] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: Bootstrap starting.
I20260925 11:01:26.879357 861 tablet_bootstrap.cc:492] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: Bootstrap starting.
I20260925 11:01:26.880119 860 tablet_bootstrap.cc:492] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: Bootstrap starting.
I20260925 11:01:26.880446 859 tablet_bootstrap.cc:654] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:26.881743 859 log.cc:824] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:26.883841 861 tablet_bootstrap.cc:654] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:26.884732 860 tablet_bootstrap.cc:654] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:26.885082 861 log.cc:824] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:26.885666 860 log.cc:824] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:26.888018 861 tablet_bootstrap.cc:492] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: No bootstrap required, opened a new log
I20260925 11:01:26.888278 861 ts_tablet_manager.cc:1397] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 11:01:26.888931 859 tablet_bootstrap.cc:492] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: No bootstrap required, opened a new log
I20260925 11:01:26.888931 860 tablet_bootstrap.cc:492] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: No bootstrap required, opened a new log
I20260925 11:01:26.889178 859 ts_tablet_manager.cc:1397] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.005s
I20260925 11:01:26.889178 860 ts_tablet_manager.cc:1397] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260925 11:01:26.895495 861 raft_consensus.cc:348] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } }
I20260925 11:01:26.895839 861 raft_consensus.cc:374] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:26.895946 861 raft_consensus.cc:729] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7456b6e5a70240dcb20f7a32ddbd79da, State: Initialized, Role: FOLLOWER
I20260925 11:01:26.896550 861 consensus_queue.cc:260] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } }
I20260925 11:01:26.897636 696 heartbeater.cc:503] Master 127.24.159.254:36441 was elected leader, sending a full tablet report...
I20260925 11:01:26.898661 861 ts_tablet_manager.cc:1428] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260925 11:01:26.899256 859 raft_consensus.cc:348] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } }
I20260925 11:01:26.899394 860 raft_consensus.cc:348] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } }
I20260925 11:01:26.899710 859 raft_consensus.cc:374] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:26.899734 860 raft_consensus.cc:374] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:26.899811 860 raft_consensus.cc:729] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf778878acfe42ff82d92a410c4d082d, State: Initialized, Role: FOLLOWER
I20260925 11:01:26.899811 859 raft_consensus.cc:729] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2f37558b864d6a85b3fce37215eb60, State: Initialized, Role: FOLLOWER
I20260925 11:01:26.900413 859 consensus_queue.cc:260] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } }
I20260925 11:01:26.900403 860 consensus_queue.cc:260] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } }
I20260925 11:01:26.902099 859 ts_tablet_manager.cc:1428] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260925 11:01:26.903175 835 heartbeater.cc:503] Master 127.24.159.254:36441 was elected leader, sending a full tablet report...
I20260925 11:01:26.904736 860 ts_tablet_manager.cc:1428] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
W20260925 11:01:27.006569 836 tablet.cc:2369] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:01:27.011335 697 tablet.cc:2369] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:01:27.031277 558 tablet.cc:2369] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:01:27.065124 865 raft_consensus.cc:482] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:27.065410 865 raft_consensus.cc:504] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } }
I20260925 11:01:27.067106 865 leader_election.cc:288] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953), 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785)
I20260925 11:01:27.073359 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.067601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:47299 (local address 127.24.159.193:43785)
0925 11:01:27.067776 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.067781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.068377 (+ 596us) server_negotiation.cc:407] Connection header received
0925 11:01:27.068576 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.068582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.068651 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.068737 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:27.069250 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.069941 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.071777 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.072513 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.072659 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.072729 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.073016 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.073135 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:01:27.073436 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.067545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:51315 (local address 127.24.159.195:34953)
0925 11:01:27.067736 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.067743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.068766 (+ 1023us) server_negotiation.cc:407] Connection header received
0925 11:01:27.068917 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.068921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.068978 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.069049 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:27.069554 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.070291 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.071921 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.072628 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.072778 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.072847 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.073091 (+ 244us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.073205 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:01:27.073546 868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.067610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43785 (local address 127.24.159.194:47299)
0925 11:01:27.068261 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:27.068280 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:27.068443 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:27.068774 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:27.068778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:27.068803 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:27.069119 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.069127 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.070067 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.070071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.071628 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.071640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.072714 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.072720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.072870 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.072900 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 11:01:27.072979 (+ 79us) client_negotiation.cc:240] Negotiation successful
0925 11:01:27.073102 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":66,"thread_start_us":356,"threads_started":1}
I20260925 11:01:27.073583 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.067633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:34953 (local address 127.24.159.194:51315)
0925 11:01:27.068640 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:27.068655 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:27.068794 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:27.069092 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:27.069097 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:27.069122 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:27.069420 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.069428 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.070416 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.070420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.071782 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.071794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.072831 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.072838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.072960 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.072982 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 11:01:27.073056 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 11:01:27.073171 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":3,"spinlock_wait_cycles":38272,"thread_start_us":725,"threads_started":1}
I20260925 11:01:27.074556 507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b99ef5cf8e41e6a1f130972e61dcb0" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2f37558b864d6a85b3fce37215eb60" is_pre_election: true
I20260925 11:01:27.074550 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b99ef5cf8e41e6a1f130972e61dcb0" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d" is_pre_election: true
I20260925 11:01:27.075124 507 raft_consensus.cc:2454] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 0.
I20260925 11:01:27.075136 790 raft_consensus.cc:2454] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 0.
I20260925 11:01:27.075820 580 leader_election.cc:302] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7456b6e5a70240dcb20f7a32ddbd79da, cf778878acfe42ff82d92a410c4d082d; no voters:
I20260925 11:01:27.076205 865 raft_consensus.cc:2781] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:27.076325 865 raft_consensus.cc:482] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:27.076431 865 raft_consensus.cc:3037] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.078570 865 raft_consensus.cc:504] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } }
I20260925 11:01:27.079113 865 leader_election.cc:288] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 election: Requested vote from peers cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953), 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785)
I20260925 11:01:27.079540 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b99ef5cf8e41e6a1f130972e61dcb0" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d"
I20260925 11:01:27.079571 507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1b99ef5cf8e41e6a1f130972e61dcb0" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2f37558b864d6a85b3fce37215eb60"
I20260925 11:01:27.079743 790 raft_consensus.cc:3037] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.079743 507 raft_consensus.cc:3037] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.082777 507 raft_consensus.cc:2454] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 1.
I20260925 11:01:27.082777 790 raft_consensus.cc:2454] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 1.
I20260925 11:01:27.083159 580 leader_election.cc:302] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2c2f37558b864d6a85b3fce37215eb60, 7456b6e5a70240dcb20f7a32ddbd79da, cf778878acfe42ff82d92a410c4d082d; no voters:
I20260925 11:01:27.083472 865 raft_consensus.cc:2781] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:27.083911 865 raft_consensus.cc:686] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 LEADER]: Becoming Leader. State: Replica: 7456b6e5a70240dcb20f7a32ddbd79da, State: Running, Role: LEADER
I20260925 11:01:27.084367 865 consensus_queue.cc:237] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } }
I20260925 11:01:27.088665 349 catalog_manager.cc:5495] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da reported cstate change: term changed from 0 to 1, leader changed from <none> to 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194). New cstate: current_term: 1 leader_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } health_report { overall_health: HEALTHY } } }
I20260925 11:01:27.155961 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.151255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.24.159.194:44161)
0925 11:01:27.151467 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.151472 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.151531 (+ 59us) server_negotiation.cc:407] Connection header received
0925 11:01:27.151717 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.151721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.151800 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.151921 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:27.152189 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.152891 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.154098 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.154530 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.154619 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.154963 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:27.155495 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:27.155598 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.155634 (+ 36us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.155691 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:01:27.163823 507 raft_consensus.cc:1261] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Refusing update from remote peer 7456b6e5a70240dcb20f7a32ddbd79da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:27.163926 790 raft_consensus.cc:1261] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Refusing update from remote peer 7456b6e5a70240dcb20f7a32ddbd79da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:27.164805 870 consensus_queue.cc:1035] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:27.165203 865 consensus_queue.cc:1035] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:27.180676 873 mvcc.cc:205] Tried to move back new op lower bound from 7333208421013303296 to 7333208420701028352. Current Snapshot: MvccSnapshot[applied={T|T < 7333208421013303296}]
I20260925 11:01:27.181254 874 mvcc.cc:205] Tried to move back new op lower bound from 7333208421013303296 to 7333208420701028352. Current Snapshot: MvccSnapshot[applied={T|T < 7333208421013303296}]
I20260925 11:01:27.194764 349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:27.194994 349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:27.198498 349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68: Sending DeleteTablet for 3 replicas of tablet c1b99ef5cf8e41e6a1f130972e61dcb0
I20260925 11:01:27.199522 765 tablet_service.cc:1514] Processing DeleteTablet for tablet c1b99ef5cf8e41e6a1f130972e61dcb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:27 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38902
I20260925 11:01:27.199690 487 tablet_service.cc:1514] Processing DeleteTablet for tablet c1b99ef5cf8e41e6a1f130972e61dcb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:27 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35690
I20260925 11:01:27.200579 626 tablet_service.cc:1514] Processing DeleteTablet for tablet c1b99ef5cf8e41e6a1f130972e61dcb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:27 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:56258
I20260925 11:01:27.200591 349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:01:27.201251 349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:27.206012 885 tablet_replica.cc:330] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: stopping tablet replica
I20260925 11:01:27.206480 885 raft_consensus.cc:2229] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:01:27.207510 885 raft_consensus.cc:2258] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:27.208954 764 tablet_service.cc:1467] Processing CreateTablet for tablet 4a055c1b7ae04ad793661ff1d1577c11 (DEFAULT_TABLE table=test-table [id=085111a1084c4caeb724619faf1e5e57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:27.209213 486 tablet_service.cc:1467] Processing CreateTablet for tablet 4a055c1b7ae04ad793661ff1d1577c11 (DEFAULT_TABLE table=test-table [id=085111a1084c4caeb724619faf1e5e57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:27.209481 764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a055c1b7ae04ad793661ff1d1577c11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:27.209692 486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a055c1b7ae04ad793661ff1d1577c11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:27.214423 626 tablet_service.cc:1467] Processing CreateTablet for tablet 4a055c1b7ae04ad793661ff1d1577c11 (DEFAULT_TABLE table=test-table [id=085111a1084c4caeb724619faf1e5e57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:27.214793 885 ts_tablet_manager.cc:1905] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:27.214885 626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a055c1b7ae04ad793661ff1d1577c11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:27.216183 860 tablet_bootstrap.cc:492] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: Bootstrap starting.
I20260925 11:01:27.216621 884 tablet_replica.cc:330] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: stopping tablet replica
I20260925 11:01:27.217623 884 raft_consensus.cc:2229] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:27.217732 860 tablet_bootstrap.cc:654] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:27.218147 884 raft_consensus.cc:2258] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:27.220645 883 tablet_replica.cc:330] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: stopping tablet replica
I20260925 11:01:27.221067 883 raft_consensus.cc:2229] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:27.221558 883 raft_consensus.cc:2258] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:27.221864 859 tablet_bootstrap.cc:492] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: Bootstrap starting.
I20260925 11:01:27.223292 859 tablet_bootstrap.cc:654] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:27.226065 883 ts_tablet_manager.cc:1905] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:27.227759 885 ts_tablet_manager.cc:1918] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:27.233973 883 ts_tablet_manager.cc:1918] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:27.234205 883 log.cc:1196] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/c1b99ef5cf8e41e6a1f130972e61dcb0
I20260925 11:01:27.234812 883 ts_tablet_manager.cc:1939] T c1b99ef5cf8e41e6a1f130972e61dcb0 P cf778878acfe42ff82d92a410c4d082d: Deleting consensus metadata
I20260925 11:01:27.233596 861 tablet_bootstrap.cc:492] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: Bootstrap starting.
I20260925 11:01:27.235651 860 tablet_bootstrap.cc:492] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: No bootstrap required, opened a new log
I20260925 11:01:27.235792 860 ts_tablet_manager.cc:1397] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20260925 11:01:27.236562 861 tablet_bootstrap.cc:654] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:27.236683 337 catalog_manager.cc:4841] TS cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953): tablet c1b99ef5cf8e41e6a1f130972e61dcb0 (table test-table [id=bb0ae8d3ba704abc97a893c6eaa39f4a]) successfully deleted
I20260925 11:01:27.236943 884 ts_tablet_manager.cc:1905] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:27.237047 860 raft_consensus.cc:348] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.237257 860 raft_consensus.cc:374] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:27.237339 860 raft_consensus.cc:729] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf778878acfe42ff82d92a410c4d082d, State: Initialized, Role: FOLLOWER
I20260925 11:01:27.237591 860 consensus_queue.cc:260] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.238620 859 tablet_bootstrap.cc:492] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: No bootstrap required, opened a new log
I20260925 11:01:27.238832 859 ts_tablet_manager.cc:1397] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.001s
I20260925 11:01:27.238868 861 tablet_bootstrap.cc:492] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: No bootstrap required, opened a new log
I20260925 11:01:27.239004 861 ts_tablet_manager.cc:1397] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260925 11:01:27.239748 859 raft_consensus.cc:348] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.239756 861 raft_consensus.cc:348] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.239955 861 raft_consensus.cc:374] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:27.240049 861 raft_consensus.cc:729] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7456b6e5a70240dcb20f7a32ddbd79da, State: Initialized, Role: FOLLOWER
I20260925 11:01:27.240084 859 raft_consensus.cc:374] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:27.240276 859 raft_consensus.cc:729] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2f37558b864d6a85b3fce37215eb60, State: Initialized, Role: FOLLOWER
I20260925 11:01:27.240281 861 consensus_queue.cc:260] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.240548 860 ts_tablet_manager.cc:1428] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 11:01:27.240855 861 ts_tablet_manager.cc:1428] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260925 11:01:27.241117 885 log.cc:1196] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/c1b99ef5cf8e41e6a1f130972e61dcb0
I20260925 11:01:27.241612 885 ts_tablet_manager.cc:1939] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting consensus metadata
I20260925 11:01:27.243577 335 catalog_manager.cc:4841] TS 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194:44161): tablet c1b99ef5cf8e41e6a1f130972e61dcb0 (table test-table [id=bb0ae8d3ba704abc97a893c6eaa39f4a]) successfully deleted
I20260925 11:01:27.243865 884 ts_tablet_manager.cc:1918] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:27.244086 884 log.cc:1196] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/c1b99ef5cf8e41e6a1f130972e61dcb0
I20260925 11:01:27.244244 859 consensus_queue.cc:260] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.245050 884 ts_tablet_manager.cc:1939] T c1b99ef5cf8e41e6a1f130972e61dcb0 P 2c2f37558b864d6a85b3fce37215eb60: Deleting consensus metadata
I20260925 11:01:27.245893 859 ts_tablet_manager.cc:1428] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260925 11:01:27.246868 335 catalog_manager.cc:4841] TS 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785): tablet c1b99ef5cf8e41e6a1f130972e61dcb0 (table test-table [id=bb0ae8d3ba704abc97a893c6eaa39f4a]) successfully deleted
I20260925 11:01:27.277565 865 raft_consensus.cc:482] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:27.277788 865 raft_consensus.cc:504] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.278504 865 leader_election.cc:288] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785), cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953)
I20260925 11:01:27.279001 507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a055c1b7ae04ad793661ff1d1577c11" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2f37558b864d6a85b3fce37215eb60" is_pre_election: true
I20260925 11:01:27.279122 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a055c1b7ae04ad793661ff1d1577c11" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d" is_pre_election: true
I20260925 11:01:27.279253 507 raft_consensus.cc:2454] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 0.
I20260925 11:01:27.279332 790 raft_consensus.cc:2454] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 0.
I20260925 11:01:27.279738 578 leader_election.cc:302] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2c2f37558b864d6a85b3fce37215eb60, 7456b6e5a70240dcb20f7a32ddbd79da, cf778878acfe42ff82d92a410c4d082d; no voters:
I20260925 11:01:27.280049 865 raft_consensus.cc:2781] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:27.280170 865 raft_consensus.cc:482] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:27.280273 865 raft_consensus.cc:3037] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.282717 865 raft_consensus.cc:504] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.283746 507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a055c1b7ae04ad793661ff1d1577c11" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2f37558b864d6a85b3fce37215eb60"
I20260925 11:01:27.283746 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a055c1b7ae04ad793661ff1d1577c11" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d"
I20260925 11:01:27.283964 507 raft_consensus.cc:3037] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.283964 790 raft_consensus.cc:3037] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.286743 790 raft_consensus.cc:2454] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 1.
I20260925 11:01:27.286785 507 raft_consensus.cc:2454] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 1.
I20260925 11:01:27.287195 578 leader_election.cc:302] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2f37558b864d6a85b3fce37215eb60, 7456b6e5a70240dcb20f7a32ddbd79da; no voters:
I20260925 11:01:27.287482 870 raft_consensus.cc:2781] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:27.287676 870 raft_consensus.cc:686] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 LEADER]: Becoming Leader. State: Replica: 7456b6e5a70240dcb20f7a32ddbd79da, State: Running, Role: LEADER
I20260925 11:01:27.287981 870 consensus_queue.cc:237] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.290583 349 catalog_manager.cc:5495] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da reported cstate change: term changed from 0 to 1, leader changed from <none> to 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194). New cstate: current_term: 1 leader_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } health_report { overall_health: UNKNOWN } } }
I20260925 11:01:27.287658 865 leader_election.cc:288] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 election: Requested vote from peers 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785), cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953)
Sep 25 11:01:27 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334085, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.254@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:27.359066 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.344440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51766 (local address 127.24.159.254:36441)
0925 11:01:27.345318 (+ 878us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.345328 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.345581 (+ 253us) server_negotiation.cc:407] Connection header received
0925 11:01:27.345736 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.345741 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.345819 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.345948 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:27.347173 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.348407 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.349622 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.350296 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.350411 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.353914 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:27.353919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:27.353923 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:27.353952 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:27.356375 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:27.357197 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:27.357203 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:27.357207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:27.357341 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:27.357728 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:27.357731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:27.357733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:27.357980 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:27.358083 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.358600 (+ 517us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.358776 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":427,"threads_started":1}
I20260925 11:01:27.359422 895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.344385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36441 (local address 127.0.0.1:51766)
0925 11:01:27.345488 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:27.345505 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:27.345629 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:27.345974 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:27.345977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:27.346754 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:27.347020 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.347029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.348546 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.348550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.349492 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.349503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.350462 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.350467 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.350602 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.351215 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:27.351239 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:27.353718 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:27.356504 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:27.356519 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:27.356527 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:27.357076 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:27.357444 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:27.357457 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:27.357461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:27.357632 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:27.358096 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:27.358105 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:27.358343 (+ 238us) client_negotiation.cc:769] Sending connection context
0925 11:01:27.358532 (+ 189us) client_negotiation.cc:240] Negotiation successful
0925 11:01:27.358960 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":105,"thread_start_us":731,"threads_started":1}
I20260925 11:01:27.364688 349 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:51766:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260925 11:01:27.365899 349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:27.372803 486 tablet_service.cc:1467] Processing CreateTablet for tablet 12827dfcca634bc5b0c11ee07bf5533d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fbef15f31311417aa46fba01d2d35c6b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 11:01:27.373387 486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12827dfcca634bc5b0c11ee07bf5533d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:27.373307 765 tablet_service.cc:1467] Processing CreateTablet for tablet 12827dfcca634bc5b0c11ee07bf5533d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fbef15f31311417aa46fba01d2d35c6b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 11:01:27.373311 626 tablet_service.cc:1467] Processing CreateTablet for tablet 12827dfcca634bc5b0c11ee07bf5533d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fbef15f31311417aa46fba01d2d35c6b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260925 11:01:27.373838 765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12827dfcca634bc5b0c11ee07bf5533d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:27.373842 626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12827dfcca634bc5b0c11ee07bf5533d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:27.380931 861 tablet_bootstrap.cc:492] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: Bootstrap starting.
I20260925 11:01:27.381735 859 tablet_bootstrap.cc:492] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: Bootstrap starting.
I20260925 11:01:27.382699 861 tablet_bootstrap.cc:654] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:27.383157 859 tablet_bootstrap.cc:654] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:27.383657 860 tablet_bootstrap.cc:492] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: Bootstrap starting.
I20260925 11:01:27.385207 860 tablet_bootstrap.cc:654] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:27.385814 859 tablet_bootstrap.cc:492] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: No bootstrap required, opened a new log
I20260925 11:01:27.385926 859 ts_tablet_manager.cc:1397] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 11:01:27.386590 859 raft_consensus.cc:348] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.386780 859 raft_consensus.cc:374] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:27.386855 859 raft_consensus.cc:729] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2f37558b864d6a85b3fce37215eb60, State: Initialized, Role: FOLLOWER
I20260925 11:01:27.387176 859 consensus_queue.cc:260] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.387678 866 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.388022 866 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:01:27.388823 860 tablet_bootstrap.cc:492] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: No bootstrap required, opened a new log
I20260925 11:01:27.388988 860 ts_tablet_manager.cc:1397] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260925 11:01:27.389652 860 raft_consensus.cc:348] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.389875 860 raft_consensus.cc:374] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:27.389956 860 raft_consensus.cc:729] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf778878acfe42ff82d92a410c4d082d, State: Initialized, Role: FOLLOWER
I20260925 11:01:27.390237 860 consensus_queue.cc:260] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.390655 867 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.390964 867 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:01:27.390966 859 ts_tablet_manager.cc:1428] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260925 11:01:27.392295 860 ts_tablet_manager.cc:1428] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260925 11:01:27.395776 861 tablet_bootstrap.cc:492] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: No bootstrap required, opened a new log
I20260925 11:01:27.395953 861 ts_tablet_manager.cc:1397] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.008s
I20260925 11:01:27.396716 861 raft_consensus.cc:348] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.396971 861 raft_consensus.cc:374] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:27.397055 861 raft_consensus.cc:729] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7456b6e5a70240dcb20f7a32ddbd79da, State: Initialized, Role: FOLLOWER
I20260925 11:01:27.397284 861 consensus_queue.cc:260] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.397938 861 ts_tablet_manager.cc:1428] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260925 11:01:27.397823 865 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.398098 865 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:01:27.456728 866 raft_consensus.cc:482] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:27.456964 866 raft_consensus.cc:504] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.458834 866 leader_election.cc:288] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194:44161), cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953)
I20260925 11:01:27.466542 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.459655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:33533 (local address 127.24.159.195:34953)
0925 11:01:27.459819 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.459824 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.460540 (+ 716us) server_negotiation.cc:407] Connection header received
0925 11:01:27.460795 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.460800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.460872 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.460932 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:27.461739 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.462814 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.464671 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.465577 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.465729 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.465799 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.466123 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.466274 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:01:27.466642 901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.459166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:34953 (local address 127.24.159.193:33533)
0925 11:01:27.460421 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:27.460444 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:27.460645 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:27.461117 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:27.461122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:27.461144 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:27.461565 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.461576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.462917 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.462922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.464513 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.464528 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.465778 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.465784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.465952 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.465988 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 11:01:27.466086 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 11:01:27.466240 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1141,"mutex_wait_us":93,"thread_start_us":1699,"threads_started":1}
I20260925 11:01:27.467473 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.459422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:58161 (local address 127.24.159.194:44161)
0925 11:01:27.459618 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.459624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.460069 (+ 445us) server_negotiation.cc:407] Connection header received
0925 11:01:27.460376 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.460382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.460460 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.460520 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:27.461408 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.462613 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.465211 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.466351 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.466508 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.466579 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.467073 (+ 494us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.467210 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 11:01:27.467629 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12827dfcca634bc5b0c11ee07bf5533d" candidate_uuid: "2c2f37558b864d6a85b3fce37215eb60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d" is_pre_election: true
I20260925 11:01:27.467888 790 raft_consensus.cc:2454] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c2f37558b864d6a85b3fce37215eb60 in term 0.
I20260925 11:01:27.468122 900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.459106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:44161 (local address 127.24.159.193:58161)
0925 11:01:27.459969 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:27.459988 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:27.460192 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:27.460718 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:27.460722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:27.460749 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:27.461228 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.461237 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.462766 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.462772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.465050 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.465066 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.466735 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.466740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.466899 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.466933 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 11:01:27.467652 (+ 719us) client_negotiation.cc:240] Negotiation successful
0925 11:01:27.467781 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":81,"thread_start_us":717,"threads_started":1}
I20260925 11:01:27.468454 440 leader_election.cc:302] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2f37558b864d6a85b3fce37215eb60, cf778878acfe42ff82d92a410c4d082d; no voters:
I20260925 11:01:27.468907 866 raft_consensus.cc:2781] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:27.469027 866 raft_consensus.cc:482] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:27.469127 866 raft_consensus.cc:3037] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.469033 646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12827dfcca634bc5b0c11ee07bf5533d" candidate_uuid: "2c2f37558b864d6a85b3fce37215eb60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" is_pre_election: true
I20260925 11:01:27.469499 646 raft_consensus.cc:2454] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c2f37558b864d6a85b3fce37215eb60 in term 0.
I20260925 11:01:27.471442 866 raft_consensus.cc:504] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.472007 866 leader_election.cc:288] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [CANDIDATE]: Term 1 election: Requested vote from peers 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194:44161), cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953)
I20260925 11:01:27.472519 646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12827dfcca634bc5b0c11ee07bf5533d" candidate_uuid: "2c2f37558b864d6a85b3fce37215eb60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7456b6e5a70240dcb20f7a32ddbd79da"
I20260925 11:01:27.472494 790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12827dfcca634bc5b0c11ee07bf5533d" candidate_uuid: "2c2f37558b864d6a85b3fce37215eb60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d"
I20260925 11:01:27.472707 646 raft_consensus.cc:3037] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.472718 790 raft_consensus.cc:3037] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.475574 790 raft_consensus.cc:2454] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c2f37558b864d6a85b3fce37215eb60 in term 1.
I20260925 11:01:27.475837 646 raft_consensus.cc:2454] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c2f37558b864d6a85b3fce37215eb60 in term 1.
I20260925 11:01:27.476238 438 leader_election.cc:302] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2f37558b864d6a85b3fce37215eb60, 7456b6e5a70240dcb20f7a32ddbd79da; no voters:
I20260925 11:01:27.476557 866 raft_consensus.cc:2781] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:27.477005 866 raft_consensus.cc:686] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [term 1 LEADER]: Becoming Leader. State: Replica: 2c2f37558b864d6a85b3fce37215eb60, State: Running, Role: LEADER
I20260925 11:01:27.477432 866 consensus_queue.cc:237] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.480607 902 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: TxnStatusTablet state changed. Reason: New leader 2c2f37558b864d6a85b3fce37215eb60. Latest consensus state: current_term: 1 leader_uuid: "2c2f37558b864d6a85b3fce37215eb60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.480953 902 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: This TxnStatusTablet replica's current role is: LEADER
I20260925 11:01:27.481096 348 catalog_manager.cc:5495] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193). New cstate: current_term: 1 leader_uuid: "2c2f37558b864d6a85b3fce37215eb60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } health_report { overall_health: UNKNOWN } } }
I20260925 11:01:27.481700 904 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260925 11:01:27.481922 904 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260925 11:01:27.521297 895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.513715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43785 (local address 127.0.0.1:35704)
0925 11:01:27.514099 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:27.514118 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:27.514289 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:27.514851 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:27.514857 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:27.514870 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:01:27.515075 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.515088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.516646 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.516652 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.518318 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.518335 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.519314 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.519321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.519555 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.519632 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:01:27.520734 (+ 1102us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:01:27.520760 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 11:01:27.520849 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 11:01:27.521002 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229}
I20260925 11:01:27.521303 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.513755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.24.159.193:43785)
0925 11:01:27.513955 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.513961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.514223 (+ 262us) server_negotiation.cc:407] Connection header received
0925 11:01:27.514532 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.514539 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.514648 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.514831 (+ 183us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:27.515219 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.516507 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.518478 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.519133 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.519269 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.519852 (+ 583us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:27.520603 (+ 751us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:27.520761 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.520872 (+ 111us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.520963 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 11:01:27.526124 790 raft_consensus.cc:1261] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Refusing update from remote peer 2c2f37558b864d6a85b3fce37215eb60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:27.526688 646 raft_consensus.cc:1261] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Refusing update from remote peer 2c2f37558b864d6a85b3fce37215eb60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:27.527083 866 consensus_queue.cc:1035] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:27.527765 902 consensus_queue.cc:1035] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:27.538846 867 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: TxnStatusTablet state changed. Reason: New leader 2c2f37558b864d6a85b3fce37215eb60. Latest consensus state: current_term: 1 leader_uuid: "2c2f37558b864d6a85b3fce37215eb60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.539124 867 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:01:27.543208 902 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c2f37558b864d6a85b3fce37215eb60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.543484 902 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: This TxnStatusTablet replica's current role is: LEADER
I20260925 11:01:27.543654 875 mvcc.cc:205] Tried to move back new op lower bound from 7333208422499540992 to 7333208422312128512. Current Snapshot: MvccSnapshot[applied={T|T < 7333208422499540992}]
I20260925 11:01:27.546379 865 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: TxnStatusTablet state changed. Reason: New leader 2c2f37558b864d6a85b3fce37215eb60. Latest consensus state: current_term: 1 leader_uuid: "2c2f37558b864d6a85b3fce37215eb60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.546679 865 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:01:27.547909 902 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c2f37558b864d6a85b3fce37215eb60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.548149 902 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P 2c2f37558b864d6a85b3fce37215eb60: This TxnStatusTablet replica's current role is: LEADER
I20260925 11:01:27.552495 865 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c2f37558b864d6a85b3fce37215eb60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.552743 865 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P 7456b6e5a70240dcb20f7a32ddbd79da: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:01:27.554463 867 tablet_replica.cc:439] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c2f37558b864d6a85b3fce37215eb60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } } }
I20260925 11:01:27.554688 867 tablet_replica.cc:442] T 12827dfcca634bc5b0c11ee07bf5533d P cf778878acfe42ff82d92a410c4d082d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260925 11:01:27.575891 856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.567989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:59589 (local address 127.24.159.193:43785)
0925 11:01:27.568181 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.568188 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.568959 (+ 771us) server_negotiation.cc:407] Connection header received
0925 11:01:27.569147 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.569152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.569246 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.569366 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:27.569972 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.571188 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.573442 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.574144 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.574312 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.575004 (+ 692us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:27.575142 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:27.575238 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.575491 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.575581 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 11:01:27.576040 911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.568077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43785 (local address 127.24.159.194:59589)
0925 11:01:27.568839 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:27.568859 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:27.569008 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:27.569389 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:27.569401 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:27.569414 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:01:27.569794 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.569807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.571345 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.571351 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.573278 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.573294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.574517 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.574524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.574729 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.574813 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:01:27.575263 (+ 450us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:01:27.575275 (+ 12us) client_negotiation.cc:769] Sending connection context
0925 11:01:27.575419 (+ 144us) client_negotiation.cc:240] Negotiation successful
0925 11:01:27.575650 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":412,"threads_started":1}
I20260925 11:01:27.587090 507 raft_consensus.cc:1261] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Refusing update from remote peer 7456b6e5a70240dcb20f7a32ddbd79da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:27.587073 790 raft_consensus.cc:1261] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Refusing update from remote peer 7456b6e5a70240dcb20f7a32ddbd79da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:27.587728 870 consensus_queue.cc:1035] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:27.588201 865 consensus_queue.cc:1035] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:27.647161 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.640977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:58255 (local address 127.24.159.194:44161)
0925 11:01:27.641167 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.641172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.641306 (+ 134us) server_negotiation.cc:407] Connection header received
0925 11:01:27.641568 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.641574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.641644 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.641717 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:27.642259 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.643220 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.645261 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.645851 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.645994 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.646486 (+ 492us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:27.646588 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:27.646666 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.646784 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.646858 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 11:01:27.647312 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.640557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:44161 (local address 127.24.159.193:58255)
0925 11:01:27.641213 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:27.641232 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:27.641432 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:27.641752 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:27.641756 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:27.641769 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:01:27.642088 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.642096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.643341 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.643346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.645137 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:27.645152 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.646021 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.646027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:27.646250 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.646316 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:01:27.646681 (+ 365us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:01:27.646692 (+ 11us) client_negotiation.cc:769] Sending connection context
0925 11:01:27.646763 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 11:01:27.646940 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":622,"threads_started":1}
I20260925 11:01:27.684513 349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:27.684756 349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:27.688081 349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68: Sending DeleteTablet for 3 replicas of tablet 4a055c1b7ae04ad793661ff1d1577c11
I20260925 11:01:27.688990 626 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a055c1b7ae04ad793661ff1d1577c11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:27 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:56258
I20260925 11:01:27.689060 765 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a055c1b7ae04ad793661ff1d1577c11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:27 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38902
I20260925 11:01:27.689257 885 tablet_replica.cc:330] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: stopping tablet replica
I20260925 11:01:27.689450 885 raft_consensus.cc:2229] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:01:27.689671 486 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a055c1b7ae04ad793661ff1d1577c11 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:27 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35690
I20260925 11:01:27.689776 885 raft_consensus.cc:2258] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:27.690084 883 tablet_replica.cc:330] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: stopping tablet replica
I20260925 11:01:27.690290 883 raft_consensus.cc:2229] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:27.690339 884 tablet_replica.cc:330] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: stopping tablet replica
I20260925 11:01:27.690466 883 raft_consensus.cc:2258] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:27.690510 884 raft_consensus.cc:2229] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:27.690666 884 raft_consensus.cc:2258] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:27.691810 884 ts_tablet_manager.cc:1905] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:27.691910 885 ts_tablet_manager.cc:1905] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:27.699239 884 ts_tablet_manager.cc:1918] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:01:27.699472 884 log.cc:1196] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/4a055c1b7ae04ad793661ff1d1577c11
I20260925 11:01:27.699949 884 ts_tablet_manager.cc:1939] T 4a055c1b7ae04ad793661ff1d1577c11 P 2c2f37558b864d6a85b3fce37215eb60: Deleting consensus metadata
I20260925 11:01:27.701124 335 catalog_manager.cc:4841] TS 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785): tablet 4a055c1b7ae04ad793661ff1d1577c11 (table test-table [id=085111a1084c4caeb724619faf1e5e57]) successfully deleted
I20260925 11:01:27.709069 885 ts_tablet_manager.cc:1918] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:01:27.709358 885 log.cc:1196] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/4a055c1b7ae04ad793661ff1d1577c11
I20260925 11:01:27.709821 885 ts_tablet_manager.cc:1939] T 4a055c1b7ae04ad793661ff1d1577c11 P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting consensus metadata
I20260925 11:01:27.710850 883 ts_tablet_manager.cc:1905] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:27.716282 883 ts_tablet_manager.cc:1918] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:01:27.716434 883 log.cc:1196] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/4a055c1b7ae04ad793661ff1d1577c11
I20260925 11:01:27.716763 883 ts_tablet_manager.cc:1939] T 4a055c1b7ae04ad793661ff1d1577c11 P cf778878acfe42ff82d92a410c4d082d: Deleting consensus metadata
I20260925 11:01:27.717751 337 catalog_manager.cc:4841] TS cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953): tablet 4a055c1b7ae04ad793661ff1d1577c11 (table test-table [id=085111a1084c4caeb724619faf1e5e57]) successfully deleted
I20260925 11:01:27.719177 335 catalog_manager.cc:4841] TS 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194:44161): tablet 4a055c1b7ae04ad793661ff1d1577c11 (table test-table [id=085111a1084c4caeb724619faf1e5e57]) successfully deleted
Sep 25 11:01:27 dist-test-slave-trms krb5kdc[305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 25 11:01:27 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:27 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 11:01:27.756858 896 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:27.743736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51778 (local address 127.24.159.254:36441)
0925 11:01:27.743935 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.743942 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.744283 (+ 341us) server_negotiation.cc:407] Connection header received
0925 11:01:27.744430 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.744436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.744531 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.744643 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:27.745935 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.747027 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.748197 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.748840 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.748954 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.756616 (+ 7662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51778: BlockingRecv error: recv got EOF from 127.0.0.1:51778 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 25 11:01:27 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM
I20260925 11:01:27.773869 896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:27.760256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51784 (local address 127.24.159.254:36441)
0925 11:01:27.760431 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:27.760438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:27.760871 (+ 433us) server_negotiation.cc:407] Connection header received
0925 11:01:27.761061 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:27.761066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:27.761134 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:27.761213 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:27.762495 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.763428 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.764598 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:27.765087 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:27.765199 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:27.768924 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:27.768929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:27.768932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:27.768968 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:27.771371 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:27.771978 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:27.771983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:27.771985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:27.772101 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:27.772491 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:27.772494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:27.772497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:27.772769 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:27.773005 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:27.773307 (+ 302us) server_negotiation.cc:299] Negotiation successful
0925 11:01:27.773533 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:01:27.777578 349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:01:27.778290 349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:27.786090 486 tablet_service.cc:1467] Processing CreateTablet for tablet 0d6b8d20ba7949c7978782763998e9ac (DEFAULT_TABLE table=test-table [id=672e2dbdc2e24490bcbc295e8583ca2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:27.786572 486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d6b8d20ba7949c7978782763998e9ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:27.790591 626 tablet_service.cc:1467] Processing CreateTablet for tablet 0d6b8d20ba7949c7978782763998e9ac (DEFAULT_TABLE table=test-table [id=672e2dbdc2e24490bcbc295e8583ca2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:27.790992 626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d6b8d20ba7949c7978782763998e9ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:27.794996 765 tablet_service.cc:1467] Processing CreateTablet for tablet 0d6b8d20ba7949c7978782763998e9ac (DEFAULT_TABLE table=test-table [id=672e2dbdc2e24490bcbc295e8583ca2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:27.795388 765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d6b8d20ba7949c7978782763998e9ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:27.801251 860 tablet_bootstrap.cc:492] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: Bootstrap starting.
I20260925 11:01:27.802855 860 tablet_bootstrap.cc:654] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:27.803859 861 tablet_bootstrap.cc:492] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: Bootstrap starting.
I20260925 11:01:27.805323 861 tablet_bootstrap.cc:654] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:27.805738 859 tablet_bootstrap.cc:492] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: Bootstrap starting.
I20260925 11:01:27.807422 859 tablet_bootstrap.cc:654] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:27.823566 861 tablet_bootstrap.cc:492] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: No bootstrap required, opened a new log
I20260925 11:01:27.823832 861 ts_tablet_manager.cc:1397] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.000s
I20260925 11:01:27.824599 859 tablet_bootstrap.cc:492] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: No bootstrap required, opened a new log
I20260925 11:01:27.824802 859 ts_tablet_manager.cc:1397] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.005s
I20260925 11:01:27.824846 860 tablet_bootstrap.cc:492] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: No bootstrap required, opened a new log
I20260925 11:01:27.824657 861 raft_consensus.cc:348] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.825011 860 ts_tablet_manager.cc:1397] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.003s
I20260925 11:01:27.825119 861 raft_consensus.cc:374] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:27.825260 861 raft_consensus.cc:729] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7456b6e5a70240dcb20f7a32ddbd79da, State: Initialized, Role: FOLLOWER
I20260925 11:01:27.825579 861 consensus_queue.cc:260] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.825742 859 raft_consensus.cc:348] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.825780 860 raft_consensus.cc:348] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.826009 860 raft_consensus.cc:374] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:27.826064 859 raft_consensus.cc:374] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:27.826122 860 raft_consensus.cc:729] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf778878acfe42ff82d92a410c4d082d, State: Initialized, Role: FOLLOWER
I20260925 11:01:27.826262 859 raft_consensus.cc:729] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2f37558b864d6a85b3fce37215eb60, State: Initialized, Role: FOLLOWER
I20260925 11:01:27.826359 860 consensus_queue.cc:260] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.826542 859 consensus_queue.cc:260] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.826967 860 ts_tablet_manager.cc:1428] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 11:01:27.827672 861 ts_tablet_manager.cc:1428] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260925 11:01:27.827723 859 ts_tablet_manager.cc:1428] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260925 11:01:27.979004 865 raft_consensus.cc:482] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:27.979324 865 raft_consensus.cc:504] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.980043 865 leader_election.cc:288] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785), cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953)
I20260925 11:01:27.980628 506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6b8d20ba7949c7978782763998e9ac" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2f37558b864d6a85b3fce37215eb60" is_pre_election: true
I20260925 11:01:27.980808 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6b8d20ba7949c7978782763998e9ac" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d" is_pre_election: true
I20260925 11:01:27.980892 506 raft_consensus.cc:2454] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 0.
I20260925 11:01:27.981050 789 raft_consensus.cc:2454] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 0.
I20260925 11:01:27.981403 578 leader_election.cc:302] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2f37558b864d6a85b3fce37215eb60, 7456b6e5a70240dcb20f7a32ddbd79da; no voters:
I20260925 11:01:27.981674 865 raft_consensus.cc:2781] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:27.981772 865 raft_consensus.cc:482] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:27.981880 865 raft_consensus.cc:3037] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.984170 865 raft_consensus.cc:504] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.984753 865 leader_election.cc:288] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 election: Requested vote from peers 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785), cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953)
I20260925 11:01:27.985159 506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6b8d20ba7949c7978782763998e9ac" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2f37558b864d6a85b3fce37215eb60"
I20260925 11:01:27.985185 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d6b8d20ba7949c7978782763998e9ac" candidate_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d"
I20260925 11:01:27.985327 506 raft_consensus.cc:3037] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.985355 789 raft_consensus.cc:3037] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:27.987257 506 raft_consensus.cc:2454] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 1.
I20260925 11:01:27.987263 789 raft_consensus.cc:2454] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7456b6e5a70240dcb20f7a32ddbd79da in term 1.
I20260925 11:01:27.987639 580 leader_election.cc:302] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7456b6e5a70240dcb20f7a32ddbd79da, cf778878acfe42ff82d92a410c4d082d; no voters:
I20260925 11:01:27.987869 865 raft_consensus.cc:2781] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:27.988024 865 raft_consensus.cc:686] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 LEADER]: Becoming Leader. State: Replica: 7456b6e5a70240dcb20f7a32ddbd79da, State: Running, Role: LEADER
I20260925 11:01:27.988265 865 consensus_queue.cc:237] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:27.990733 350 catalog_manager.cc:5495] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da reported cstate change: term changed from 0 to 1, leader changed from <none> to 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194). New cstate: current_term: 1 leader_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } health_report { overall_health: UNKNOWN } } }
I20260925 11:01:28.043911 858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:28.039727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56286 (local address 127.24.159.194:44161)
0925 11:01:28.039910 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:28.039915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:28.039981 (+ 66us) server_negotiation.cc:407] Connection header received
0925 11:01:28.040132 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:28.040135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:28.040200 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:28.040284 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:28.040564 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.041258 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.042526 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.042944 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.043043 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:28.043363 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:28.043477 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:28.043540 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:28.043624 (+ 84us) server_negotiation.cc:299] Negotiation successful
0925 11:01:28.043686 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:01:28.046412 789 raft_consensus.cc:1261] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Refusing update from remote peer 7456b6e5a70240dcb20f7a32ddbd79da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:28.046412 506 raft_consensus.cc:1261] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Refusing update from remote peer 7456b6e5a70240dcb20f7a32ddbd79da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:28.047042 870 consensus_queue.cc:1035] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:28.047327 865 consensus_queue.cc:1035] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:28.065002 349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:28.065219 349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:28.067847 349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68: Sending DeleteTablet for 3 replicas of tablet 0d6b8d20ba7949c7978782763998e9ac
I20260925 11:01:28.068516 486 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d6b8d20ba7949c7978782763998e9ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:28 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35690
I20260925 11:01:28.068612 626 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d6b8d20ba7949c7978782763998e9ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:28 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:56258
I20260925 11:01:28.068612 765 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d6b8d20ba7949c7978782763998e9ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:28 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38902
I20260925 11:01:28.068923 884 tablet_replica.cc:330] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: stopping tablet replica
I20260925 11:01:28.068920 885 tablet_replica.cc:330] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: stopping tablet replica
I20260925 11:01:28.069103 884 raft_consensus.cc:2229] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:28.069108 885 raft_consensus.cc:2229] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:01:28.069216 883 tablet_replica.cc:330] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: stopping tablet replica
I20260925 11:01:28.069334 884 raft_consensus.cc:2258] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:28.069382 883 raft_consensus.cc:2229] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:28.069424 885 raft_consensus.cc:2258] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:28.069625 883 raft_consensus.cc:2258] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:28.069779 349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:01:28.070429 349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:28.070591 884 ts_tablet_manager.cc:1905] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:28.070591 885 ts_tablet_manager.cc:1905] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:28.072314 883 ts_tablet_manager.cc:1905] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:28.077111 885 ts_tablet_manager.cc:1918] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:28.077244 885 log.cc:1196] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/0d6b8d20ba7949c7978782763998e9ac
I20260925 11:01:28.077322 486 tablet_service.cc:1467] Processing CreateTablet for tablet 69a8638ea5cf4eec87f70e3c60201087 (DEFAULT_TABLE table=test-table [id=136d415bb54a4af2a3ff43a1f0b1fa24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:28.077373 626 tablet_service.cc:1467] Processing CreateTablet for tablet 69a8638ea5cf4eec87f70e3c60201087 (DEFAULT_TABLE table=test-table [id=136d415bb54a4af2a3ff43a1f0b1fa24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:28.077673 885 ts_tablet_manager.cc:1939] T 0d6b8d20ba7949c7978782763998e9ac P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting consensus metadata
I20260925 11:01:28.077775 626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a8638ea5cf4eec87f70e3c60201087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:28.078127 486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a8638ea5cf4eec87f70e3c60201087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:28.078667 883 ts_tablet_manager.cc:1918] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:28.078778 883 log.cc:1196] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/0d6b8d20ba7949c7978782763998e9ac
I20260925 11:01:28.078753 335 catalog_manager.cc:4841] TS 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194:44161): tablet 0d6b8d20ba7949c7978782763998e9ac (table test-table [id=672e2dbdc2e24490bcbc295e8583ca2d]) successfully deleted
I20260925 11:01:28.079131 883 ts_tablet_manager.cc:1939] T 0d6b8d20ba7949c7978782763998e9ac P cf778878acfe42ff82d92a410c4d082d: Deleting consensus metadata
I20260925 11:01:28.079309 884 ts_tablet_manager.cc:1918] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:28.079466 884 log.cc:1196] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/0d6b8d20ba7949c7978782763998e9ac
I20260925 11:01:28.079895 884 ts_tablet_manager.cc:1939] T 0d6b8d20ba7949c7978782763998e9ac P 2c2f37558b864d6a85b3fce37215eb60: Deleting consensus metadata
I20260925 11:01:28.080271 337 catalog_manager.cc:4841] TS cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953): tablet 0d6b8d20ba7949c7978782763998e9ac (table test-table [id=672e2dbdc2e24490bcbc295e8583ca2d]) successfully deleted
I20260925 11:01:28.080941 335 catalog_manager.cc:4841] TS 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785): tablet 0d6b8d20ba7949c7978782763998e9ac (table test-table [id=672e2dbdc2e24490bcbc295e8583ca2d]) successfully deleted
I20260925 11:01:28.081128 765 tablet_service.cc:1467] Processing CreateTablet for tablet 69a8638ea5cf4eec87f70e3c60201087 (DEFAULT_TABLE table=test-table [id=136d415bb54a4af2a3ff43a1f0b1fa24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:28.081525 765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 69a8638ea5cf4eec87f70e3c60201087. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:28.083587 861 tablet_bootstrap.cc:492] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: Bootstrap starting.
I20260925 11:01:28.084183 859 tablet_bootstrap.cc:492] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: Bootstrap starting.
I20260925 11:01:28.085044 861 tablet_bootstrap.cc:654] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:28.085716 859 tablet_bootstrap.cc:654] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:28.086925 861 tablet_bootstrap.cc:492] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: No bootstrap required, opened a new log
I20260925 11:01:28.087088 861 ts_tablet_manager.cc:1397] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260925 11:01:28.087172 860 tablet_bootstrap.cc:492] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: Bootstrap starting.
I20260925 11:01:28.087791 861 raft_consensus.cc:348] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:28.088023 861 raft_consensus.cc:374] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:28.088179 861 raft_consensus.cc:729] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7456b6e5a70240dcb20f7a32ddbd79da, State: Initialized, Role: FOLLOWER
I20260925 11:01:28.088502 859 tablet_bootstrap.cc:492] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: No bootstrap required, opened a new log
I20260925 11:01:28.088414 861 consensus_queue.cc:260] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:28.088660 859 ts_tablet_manager.cc:1397] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260925 11:01:28.088737 860 tablet_bootstrap.cc:654] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:28.089115 861 ts_tablet_manager.cc:1428] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260925 11:01:28.090076 859 raft_consensus.cc:348] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:28.090329 859 raft_consensus.cc:374] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:28.090401 859 raft_consensus.cc:729] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2f37558b864d6a85b3fce37215eb60, State: Initialized, Role: FOLLOWER
I20260925 11:01:28.090605 859 consensus_queue.cc:260] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:28.091022 859 ts_tablet_manager.cc:1428] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260925 11:01:28.091902 860 tablet_bootstrap.cc:492] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: No bootstrap required, opened a new log
I20260925 11:01:28.092064 860 ts_tablet_manager.cc:1397] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260925 11:01:28.092782 860 raft_consensus.cc:348] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:28.093031 860 raft_consensus.cc:374] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:28.093112 860 raft_consensus.cc:729] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf778878acfe42ff82d92a410c4d082d, State: Initialized, Role: FOLLOWER
I20260925 11:01:28.093303 860 consensus_queue.cc:260] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:28.093652 860 ts_tablet_manager.cc:1428] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260925 11:01:28.290351 902 raft_consensus.cc:482] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:28.290539 902 raft_consensus.cc:504] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:28.291110 902 leader_election.cc:288] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194:44161), cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953)
I20260925 11:01:28.291632 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a8638ea5cf4eec87f70e3c60201087" candidate_uuid: "2c2f37558b864d6a85b3fce37215eb60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d" is_pre_election: true
I20260925 11:01:28.291666 645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a8638ea5cf4eec87f70e3c60201087" candidate_uuid: "2c2f37558b864d6a85b3fce37215eb60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" is_pre_election: true
I20260925 11:01:28.291829 789 raft_consensus.cc:2454] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c2f37558b864d6a85b3fce37215eb60 in term 0.
I20260925 11:01:28.291911 645 raft_consensus.cc:2454] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c2f37558b864d6a85b3fce37215eb60 in term 0.
I20260925 11:01:28.292279 440 leader_election.cc:302] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2f37558b864d6a85b3fce37215eb60, cf778878acfe42ff82d92a410c4d082d; no voters:
I20260925 11:01:28.292570 902 raft_consensus.cc:2781] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:28.292654 902 raft_consensus.cc:482] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:28.292727 902 raft_consensus.cc:3037] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:28.294591 902 raft_consensus.cc:504] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:28.295100 902 leader_election.cc:288] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [CANDIDATE]: Term 1 election: Requested vote from peers 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194:44161), cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953)
I20260925 11:01:28.295508 645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a8638ea5cf4eec87f70e3c60201087" candidate_uuid: "2c2f37558b864d6a85b3fce37215eb60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7456b6e5a70240dcb20f7a32ddbd79da"
I20260925 11:01:28.295675 645 raft_consensus.cc:3037] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:28.295831 789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "69a8638ea5cf4eec87f70e3c60201087" candidate_uuid: "2c2f37558b864d6a85b3fce37215eb60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf778878acfe42ff82d92a410c4d082d"
I20260925 11:01:28.296048 789 raft_consensus.cc:3037] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:28.298645 645 raft_consensus.cc:2454] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c2f37558b864d6a85b3fce37215eb60 in term 1.
I20260925 11:01:28.298811 789 raft_consensus.cc:2454] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c2f37558b864d6a85b3fce37215eb60 in term 1.
I20260925 11:01:28.299041 438 leader_election.cc:302] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2f37558b864d6a85b3fce37215eb60, 7456b6e5a70240dcb20f7a32ddbd79da; no voters:
I20260925 11:01:28.299283 902 raft_consensus.cc:2781] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:28.299651 902 raft_consensus.cc:686] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 LEADER]: Becoming Leader. State: Replica: 2c2f37558b864d6a85b3fce37215eb60, State: Running, Role: LEADER
I20260925 11:01:28.299893 902 consensus_queue.cc:237] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } }
I20260925 11:01:28.302137 350 catalog_manager.cc:5495] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193). New cstate: current_term: 1 leader_uuid: "2c2f37558b864d6a85b3fce37215eb60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c2f37558b864d6a85b3fce37215eb60" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 } health_report { overall_health: UNKNOWN } } }
I20260925 11:01:28.337805 942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:28.333156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.24.159.193:43785)
0925 11:01:28.333625 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:28.333629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:28.333642 (+ 13us) server_negotiation.cc:407] Connection header received
0925 11:01:28.333687 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:28.333691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:28.333748 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:28.333829 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:28.334228 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.334921 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.336188 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.336622 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.336727 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:28.337136 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:28.337240 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:28.337311 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:28.337454 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 11:01:28.337527 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":201,"threads_started":1}
I20260925 11:01:28.349437 942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:28.345133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:42867 (local address 127.24.159.193:43785)
0925 11:01:28.345285 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:28.345289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:28.345618 (+ 329us) server_negotiation.cc:407] Connection header received
0925 11:01:28.345744 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:28.345748 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:28.345803 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:28.345872 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:28.346244 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.346940 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.348166 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.348539 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.348626 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:28.348934 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:28.349026 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:28.349086 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:28.349175 (+ 89us) server_negotiation.cc:299] Negotiation successful
0925 11:01:28.349227 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 11:01:28.349519 945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:28.345053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43785 (local address 127.24.159.193:42867)
0925 11:01:28.345539 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:28.345558 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:28.345659 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:28.345886 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:28.345889 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:28.345907 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0925 11:01:28.346111 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:28.346118 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.347062 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.347066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:28.348045 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:28.348056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.348670 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.348675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:28.348793 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:28.348824 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0925 11:01:28.349095 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0925 11:01:28.349102 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 11:01:28.349153 (+ 51us) client_negotiation.cc:240] Negotiation successful
0925 11:01:28.349240 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":249,"threads_started":1}
I20260925 11:01:28.356333 789 raft_consensus.cc:1261] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Refusing update from remote peer 2c2f37558b864d6a85b3fce37215eb60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:28.356302 645 raft_consensus.cc:1261] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Refusing update from remote peer 2c2f37558b864d6a85b3fce37215eb60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:28.356844 941 consensus_queue.cc:1035] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf778878acfe42ff82d92a410c4d082d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:28.357154 939 consensus_queue.cc:1035] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7456b6e5a70240dcb20f7a32ddbd79da" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:28.399075 349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:28.399281 349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:28.402146 349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6524dbf9b7274cafbfe0442612b32e68: Sending DeleteTablet for 3 replicas of tablet 69a8638ea5cf4eec87f70e3c60201087
I20260925 11:01:28.402860 486 tablet_service.cc:1514] Processing DeleteTablet for tablet 69a8638ea5cf4eec87f70e3c60201087 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:28 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35690
I20260925 11:01:28.403069 626 tablet_service.cc:1514] Processing DeleteTablet for tablet 69a8638ea5cf4eec87f70e3c60201087 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:28 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:56258
I20260925 11:01:28.403156 884 tablet_replica.cc:330] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: stopping tablet replica
I20260925 11:01:28.403348 885 tablet_replica.cc:330] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: stopping tablet replica
I20260925 11:01:28.403362 884 raft_consensus.cc:2229] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:01:28.403535 885 raft_consensus.cc:2229] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:28.403708 885 raft_consensus.cc:2258] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:28.403823 884 raft_consensus.cc:2258] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:28.404458 765 tablet_service.cc:1514] Processing DeleteTablet for tablet 69a8638ea5cf4eec87f70e3c60201087 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:28 UTC) from {username='oryx', principal='oryx/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38902
I20260925 11:01:28.404721 883 tablet_replica.cc:330] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: stopping tablet replica
I20260925 11:01:28.404907 883 raft_consensus.cc:2229] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:28.405012 884 ts_tablet_manager.cc:1905] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:28.405071 883 raft_consensus.cc:2258] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:28.405102 885 ts_tablet_manager.cc:1905] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:28.405594 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 421
I20260925 11:01:28.412213 884 ts_tablet_manager.cc:1918] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:01:28.412457 884 log.cc:1196] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/69a8638ea5cf4eec87f70e3c60201087
I20260925 11:01:28.412998 884 ts_tablet_manager.cc:1939] T 69a8638ea5cf4eec87f70e3c60201087 P 2c2f37558b864d6a85b3fce37215eb60: Deleting consensus metadata
I20260925 11:01:28.413781 883 ts_tablet_manager.cc:1905] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:28.414242 885 ts_tablet_manager.cc:1918] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:01:28.414232 335 catalog_manager.cc:4841] TS 2c2f37558b864d6a85b3fce37215eb60 (127.24.159.193:43785): tablet 69a8638ea5cf4eec87f70e3c60201087 (table test-table [id=136d415bb54a4af2a3ff43a1f0b1fa24]) successfully deleted
I20260925 11:01:28.414384 885 log.cc:1196] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/69a8638ea5cf4eec87f70e3c60201087
I20260925 11:01:28.414741 885 ts_tablet_manager.cc:1939] T 69a8638ea5cf4eec87f70e3c60201087 P 7456b6e5a70240dcb20f7a32ddbd79da: Deleting consensus metadata
I20260925 11:01:28.415833 335 catalog_manager.cc:4841] TS 7456b6e5a70240dcb20f7a32ddbd79da (127.24.159.194:44161): tablet 69a8638ea5cf4eec87f70e3c60201087 (table test-table [id=136d415bb54a4af2a3ff43a1f0b1fa24]) successfully deleted
I20260925 11:01:28.419503 883 ts_tablet_manager.cc:1918] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260925 11:01:28.419662 883 log.cc:1196] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/69a8638ea5cf4eec87f70e3c60201087
I20260925 11:01:28.420037 883 ts_tablet_manager.cc:1939] T 69a8638ea5cf4eec87f70e3c60201087 P cf778878acfe42ff82d92a410c4d082d: Deleting consensus metadata
I20260925 11:01:28.421092 337 catalog_manager.cc:4841] TS cf778878acfe42ff82d92a410c4d082d (127.24.159.195:34953): tablet 69a8638ea5cf4eec87f70e3c60201087 (table test-table [id=136d415bb54a4af2a3ff43a1f0b1fa24]) successfully deleted
Sep 25 11:01:28 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.193@KRBTEST.COM
I20260925 11:01:28.433027 942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:28.407269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35718 (local address 127.24.159.193:43785)
0925 11:01:28.411915 (+ 4646us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:28.411922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:28.416188 (+ 4266us) server_negotiation.cc:407] Connection header received
0925 11:01:28.416534 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:28.416540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:28.416624 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:28.416726 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:28.418219 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.419385 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.421250 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.421703 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.421802 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:28.426362 (+ 4560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:28.426382 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:28.426404 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:28.426452 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:28.430298 (+ 3846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:28.430918 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:28.430928 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:28.430932 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:28.431109 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:28.431432 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:28.431437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:28.431441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:28.431913 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:28.432244 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:28.432408 (+ 164us) server_negotiation.cc:299] Negotiation successful
0925 11:01:28.432625 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4387}
I20260925 11:01:28.433666 552 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:28.559868 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 421
I20260925 11:01:28.578821 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 561
Sep 25 11:01:28 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.194@KRBTEST.COM
I20260925 11:01:28.592795 952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:28.579483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.24.159.194:44161)
0925 11:01:28.579935 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:28.579940 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:28.579956 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:01:28.580010 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:28.580014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:28.580081 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:28.580169 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:28.581270 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.581958 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.583278 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.583701 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.583802 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:28.587533 (+ 3731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:28.587547 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:28.587563 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:28.587596 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:28.590265 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:28.590872 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:28.590879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:28.590882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:28.591011 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:28.591351 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:28.591355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:28.591357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:28.591697 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:28.591979 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:28.592250 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 11:01:28.592459 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":247,"threads_started":1}
I20260925 11:01:28.593431 691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:28.714018 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 561
I20260925 11:01:28.732323 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 700
Sep 25 11:01:28 dist-test-slave-trms krb5kdc[305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.159.195@KRBTEST.COM
I20260925 11:01:28.746037 954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:28.733045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.24.159.195:34953)
0925 11:01:28.733481 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:28.733485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:28.733502 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:01:28.733554 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:28.733558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:28.733616 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:28.733713 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:28.734927 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.735619 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.736740 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:28.737139 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:28.737227 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:28.741195 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:28.741211 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:28.741227 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:28.741259 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:28.743900 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:28.744473 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:28.744479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:28.744481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:28.744600 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:28.744851 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:28.744853 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:28.744855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:28.745167 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:28.745388 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:28.745555 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 11:01:28.745709 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":241,"threads_started":1}
I20260925 11:01:28.746726 830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:28.840173 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 700
I20260925 11:01:28.854293 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 318
I20260925 11:01:28.855440 390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:28.968787 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 318
2026-09-25T11:01:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5407 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:01:29 dist-test-slave-trms krb5kdc[959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:01:29 dist-test-slave-trms krb5kdc[959](info): set up 2 sockets
Sep 25 11:01:29 dist-test-slave-trms krb5kdc[959](info): commencing operation
krb5kdc: starting...
W20260925 11:01:29.975028 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.967s user 0.005s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:01:30 dist-test-slave-trms krb5kdc[959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:01:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:01:30.173640 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45617
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:45617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:30.482823 972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:30.483122 972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:30.483244 972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:30.490288 972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:30.490396 972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:30.490471 972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:30.490550 972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:30.490612 972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:30.499969 972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:45617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:30.501220 972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:30.502511 972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:30.513551 977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:30.513516 978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:30.513574 980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:30.514273 972 server_base.cc:1034] running on GCE node
I20260925 11:01:30.515057 972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:30.516467 972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:30.517666 972 hybrid_clock.cc:648] HybridClock initialized: now 1790334090517610 us; error 45 us; skew 500 ppm
I20260925 11:01:30.517993 972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-25T11:01:30Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1611 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:01:30 dist-test-slave-trms krb5kdc[986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:01:30 dist-test-slave-trms krb5kdc[986](info): set up 2 sockets
Sep 25 11:01:30 dist-test-slave-trms krb5kdc[986](info): commencing operation
krb5kdc: starting...
W20260925 11:01:31.591116 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.004s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:01:31 dist-test-slave-trms krb5kdc[986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:01:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:01:31.789502 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37217
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:37217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:32.100198 999 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:32.100486 999 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:32.100574 999 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:32.107668 999 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:32.107772 999 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:32.107817 999 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:32.107872 999 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:32.107908 999 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:32.117511 999 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:37217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:32.118778 999 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:32.120014 999 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:32.130849 1004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:32.130828 1005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:32.131049 1007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:32.131668 999 server_base.cc:1034] running on GCE node
I20260925 11:01:32.132405 999 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:32.133746 999 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:32.134950 999 hybrid_clock.cc:648] HybridClock initialized: now 1790334092134895 us; error 42 us; skew 500 ppm
I20260925 11:01:32.135270 999 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:32 dist-test-slave-trms krb5kdc[986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:32.138092 999 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:01:32.139433 999 webserver.cc:460] Webserver started at http://127.24.159.254:39815/ using document root <none> and password file <none>
I20260925 11:01:32.140081 999 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:32.140193 999 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:32.140475 999 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:32.143277 999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "60480c443e7e45168e5b312b465a50c1"
format_stamp: "Formatted at 2026-09-25 11:01:32 on dist-test-slave-trms"
server_key: "7990bc5e91042528badb09a8fd9e9594"
server_key_iv: "a74cdfa0baff62fab432a1eff8ac0c8a"
server_key_version: "encryptionkey@0"
I20260925 11:01:32.143973 999 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "60480c443e7e45168e5b312b465a50c1"
format_stamp: "Formatted at 2026-09-25 11:01:32 on dist-test-slave-trms"
server_key: "7990bc5e91042528badb09a8fd9e9594"
server_key_iv: "a74cdfa0baff62fab432a1eff8ac0c8a"
server_key_version: "encryptionkey@0"
I20260925 11:01:32.148797 999 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 11:01:32.152092 1014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:32.153326 999 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:01:32.153486 999 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "60480c443e7e45168e5b312b465a50c1"
format_stamp: "Formatted at 2026-09-25 11:01:32 on dist-test-slave-trms"
server_key: "7990bc5e91042528badb09a8fd9e9594"
server_key_iv: "a74cdfa0baff62fab432a1eff8ac0c8a"
server_key_version: "encryptionkey@0"
I20260925 11:01:32.153658 999 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:32.169382 999 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:32.172931 999 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:32.173177 999 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:32.189390 999 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:37217
I20260925 11:01:32.189404 1065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:37217 every 8 connection(s)
I20260925 11:01:32.190790 999 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:32.194309 1066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:32.196839 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 999
I20260925 11:01:32.197013 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:32.197625 25215 external_mini_cluster.cc:1428] Setting key 53ba9674bb2e0f0290f12382d7b4bfbe
I20260925 11:01:32.206554 1066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: Bootstrap starting.
I20260925 11:01:32.210047 1066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:32.211031 1066 log.cc:824] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: Log is configured to *not* fsync() on all Append() calls
Sep 25 11:01:32 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:32.214294 1066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: No bootstrap required, opened a new log
I20260925 11:01:32.220994 1069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:32.199507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40634 (local address 127.24.159.254:37217)
0925 11:01:32.200254 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:32.200269 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:32.200317 (+ 48us) server_negotiation.cc:407] Connection header received
0925 11:01:32.202480 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:32.202509 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:32.203041 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:32.203476 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:32.204653 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:32.206567 (+ 1914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:32.207769 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:32.208815 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:32.208995 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:32.212744 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:32.212755 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:32.212773 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:32.212815 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:32.216618 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:32.217266 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:32.217275 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:32.217284 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:32.217496 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:32.217787 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:32.217791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:32.217795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:32.218538 (+ 743us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:32.218802 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:32.219139 (+ 337us) server_negotiation.cc:299] Negotiation successful
0925 11:01:32.219605 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":317,"threads_started":1}
I20260925 11:01:32.223731 1066 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } }
I20260925 11:01:32.224050 1066 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:32.224123 1066 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60480c443e7e45168e5b312b465a50c1, State: Initialized, Role: FOLLOWER
I20260925 11:01:32.224591 1066 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } }
I20260925 11:01:32.224782 1066 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:32.224874 1066 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:32.224982 1066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:32.226997 1066 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } }
I20260925 11:01:32.227483 1066 leader_election.cc:302] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60480c443e7e45168e5b312b465a50c1; no voters:
I20260925 11:01:32.228044 1066 leader_election.cc:288] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:32.228140 1071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:32.228752 1071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 1 LEADER]: Becoming Leader. State: Replica: 60480c443e7e45168e5b312b465a50c1, State: Running, Role: LEADER
I20260925 11:01:32.229257 1071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } }
I20260925 11:01:32.230154 1066 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:32.232218 1072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60480c443e7e45168e5b312b465a50c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } } }
I20260925 11:01:32.232197 1073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60480c443e7e45168e5b312b465a50c1. Latest consensus state: current_term: 1 leader_uuid: "60480c443e7e45168e5b312b465a50c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } } }
I20260925 11:01:32.232630 1072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:32.232626 1073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:32.233356 1078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:32.238595 1078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:32.247655 1078 catalog_manager.cc:1347] Generated new cluster ID: 8eb8b9c55392407b8fde9ba272e06254
I20260925 11:01:32.247766 1078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:32.262440 1078 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:32.264271 1078 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:32.271359 1078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: Generated new TSK 0
I20260925 11:01:32.272398 1078 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:01:32.333428 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37217
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:32.648931 1092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:32.649210 1092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:32.649348 1092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:32.656602 1092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:32.656710 1092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:32.656870 1092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:32.666575 1092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:32.667851 1092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:32.669162 1092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:32.681559 1097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:32.681565 1100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:32.681649 1098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:32.682648 1092 server_base.cc:1034] running on GCE node
I20260925 11:01:32.683244 1092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:32.684161 1092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:32.685348 1092 hybrid_clock.cc:648] HybridClock initialized: now 1790334092685294 us; error 43 us; skew 500 ppm
I20260925 11:01:32.685644 1092 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:32 dist-test-slave-trms krb5kdc[986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:32.688560 1092 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:01:32.689981 1092 webserver.cc:460] Webserver started at http://127.24.159.193:35541/ using document root <none> and password file <none>
I20260925 11:01:32.690680 1092 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:32.690793 1092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:32.691073 1092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:32.693909 1092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "b422391743734bc6bddbc61ef7a4e9fe"
format_stamp: "Formatted at 2026-09-25 11:01:32 on dist-test-slave-trms"
server_key: "6e54ce28cb118cb3c438d0266ff5c80e"
server_key_iv: "f3596632b8fc04d68aef37c706883692"
server_key_version: "encryptionkey@0"
I20260925 11:01:32.694700 1092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "b422391743734bc6bddbc61ef7a4e9fe"
format_stamp: "Formatted at 2026-09-25 11:01:32 on dist-test-slave-trms"
server_key: "6e54ce28cb118cb3c438d0266ff5c80e"
server_key_iv: "f3596632b8fc04d68aef37c706883692"
server_key_version: "encryptionkey@0"
I20260925 11:01:32.699601 1092 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260925 11:01:32.702935 1107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:32.704246 1092 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:01:32.704391 1092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "b422391743734bc6bddbc61ef7a4e9fe"
format_stamp: "Formatted at 2026-09-25 11:01:32 on dist-test-slave-trms"
server_key: "6e54ce28cb118cb3c438d0266ff5c80e"
server_key_iv: "f3596632b8fc04d68aef37c706883692"
server_key_version: "encryptionkey@0"
I20260925 11:01:32.704582 1092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:32.718720 1092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:32.722797 1092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:32.723058 1092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:32.723963 1092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:32.725230 1092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:32.725308 1092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:32.725427 1092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:32.725481 1092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:32.754452 1092 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:41593
I20260925 11:01:32.754500 1219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:41593 every 8 connection(s)
I20260925 11:01:32.755760 1092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:32.765130 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 1092
I20260925 11:01:32.765311 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:01:32.765879 25215 external_mini_cluster.cc:1428] Setting key 447ee402e13ba699ee12fa0c45dfe224
Sep 25 11:01:32 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334092, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:32.782522 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:32.758349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:54573 (local address 127.24.159.254:37217)
0925 11:01:32.762531 (+ 4182us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:32.762537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:32.762564 (+ 27us) server_negotiation.cc:407] Connection header received
0925 11:01:32.762798 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:32.762804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:32.762877 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:32.762966 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:32.765112 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:32.765963 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:32.768029 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:32.768603 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:32.768751 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:32.777064 (+ 8313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:32.777070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:32.777073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:32.777109 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:32.779414 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:32.780173 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:32.780178 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:32.780182 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:32.780326 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:32.780758 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:32.780762 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:32.780764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:32.781051 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:32.781183 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:32.781892 (+ 709us) server_negotiation.cc:299] Negotiation successful
0925 11:01:32.782129 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4065,"thread_start_us":3974,"threads_started":1}
I20260925 11:01:32.783488 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:32.758911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:54573)
0925 11:01:32.759814 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:32.759872 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:32.762474 (+ 2602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:32.763251 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:32.763264 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:32.764022 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:32.764875 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:32.764910 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:32.766139 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:32.766144 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:32.767880 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:32.767893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:32.768830 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:32.768836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:32.769056 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:32.774228 (+ 5172us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:32.774266 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:32.776806 (+ 2540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:32.779565 (+ 2759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:32.779575 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:32.779594 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:32.780012 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:32.780437 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:32.780442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:32.780445 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:32.780643 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:32.781174 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:32.781183 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:32.781540 (+ 357us) client_negotiation.cc:769] Sending connection context
0925 11:01:32.781774 (+ 234us) client_negotiation.cc:240] Negotiation successful
0925 11:01:32.782341 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":315,"threads_started":1}
I20260925 11:01:32.785013 1220 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37217
I20260925 11:01:32.785319 1220 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
I20260925 11:01:32.786046 1220 heartbeater.cc:511] Master 127.24.159.254:37217 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
I20260925 11:01:32.788251 1031 ts_manager.cc:194] Registered new tserver with Master: b422391743734bc6bddbc61ef7a4e9fe (127.24.159.193:41593)
I20260925 11:01:32.789671 1031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:54573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:01:32.810992 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37217
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:33.123463 1226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:33.123755 1226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:33.123893 1226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:33.131078 1226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:33.131187 1226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:33.131335 1226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:33.140915 1226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:33.142170 1226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:33.143491 1226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:33.155766 1231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:33.155707 1232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:33.155807 1234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:33.156368 1226 server_base.cc:1034] running on GCE node
I20260925 11:01:33.156971 1226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:33.157848 1226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:33.159066 1226 hybrid_clock.cc:648] HybridClock initialized: now 1790334093159016 us; error 43 us; skew 500 ppm
I20260925 11:01:33.159390 1226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:33 dist-test-slave-trms krb5kdc[986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334093, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:33.162408 1226 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:01:33.163657 1226 webserver.cc:460] Webserver started at http://127.24.159.194:38907/ using document root <none> and password file <none>
I20260925 11:01:33.164296 1226 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:33.164403 1226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:33.164678 1226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:33.167469 1226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "332ce21385d14507aefd003bc596bc33"
format_stamp: "Formatted at 2026-09-25 11:01:33 on dist-test-slave-trms"
server_key: "cad96f8496d235ecdc9caad59fb9cb22"
server_key_iv: "dc254083781e059affa33dcf604fc62a"
server_key_version: "encryptionkey@0"
I20260925 11:01:33.168188 1226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "332ce21385d14507aefd003bc596bc33"
format_stamp: "Formatted at 2026-09-25 11:01:33 on dist-test-slave-trms"
server_key: "cad96f8496d235ecdc9caad59fb9cb22"
server_key_iv: "dc254083781e059affa33dcf604fc62a"
server_key_version: "encryptionkey@0"
I20260925 11:01:33.172917 1226 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 11:01:33.176194 1241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:33.177412 1226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 11:01:33.177551 1226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "332ce21385d14507aefd003bc596bc33"
format_stamp: "Formatted at 2026-09-25 11:01:33 on dist-test-slave-trms"
server_key: "cad96f8496d235ecdc9caad59fb9cb22"
server_key_iv: "dc254083781e059affa33dcf604fc62a"
server_key_version: "encryptionkey@0"
I20260925 11:01:33.177724 1226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:33.203413 1226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:33.208374 1226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:33.208662 1226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:33.209757 1226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:33.211161 1226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:33.211234 1226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:33.211316 1226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:33.211357 1226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:33.240051 1226 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:37187
I20260925 11:01:33.240118 1353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:37187 every 8 connection(s)
I20260925 11:01:33.241289 1226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:33.248755 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 1226
I20260925 11:01:33.248907 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:01:33.249305 25215 external_mini_cluster.cc:1428] Setting key e0f345aebcf81fc6f6b680ffb593e108
Sep 25 11:01:33 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334093, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:33.261220 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:33.243670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:37103 (local address 127.24.159.254:37217)
0925 11:01:33.243848 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:33.243853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:33.244997 (+ 1144us) server_negotiation.cc:407] Connection header received
0925 11:01:33.246831 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:33.246837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:33.246921 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:33.247006 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:33.248518 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.249309 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.251317 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.251700 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.251790 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:33.255677 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:33.255682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:33.255685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:33.255719 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:33.258369 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:33.259045 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:33.259051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:33.259054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:33.259208 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:33.259576 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:33.259580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:33.259584 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:33.259908 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:33.260036 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:33.260692 (+ 656us) server_negotiation.cc:299] Negotiation successful
0925 11:01:33.260924 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:01:33.262022 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:33.244153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:37103)
0925 11:01:33.244835 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:33.244886 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:33.246558 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:33.247248 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:33.247256 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:33.247774 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:33.248333 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:33.248357 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.249520 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.249525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:33.251176 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:33.251189 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.251930 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.251938 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:33.252197 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:33.252879 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:33.252909 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:33.255453 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:33.258494 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:33.258501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:33.258516 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:33.258921 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:33.259296 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:33.259299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:33.259301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:33.259479 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:33.260011 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:33.260016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:33.260326 (+ 310us) client_negotiation.cc:769] Sending connection context
0925 11:01:33.260552 (+ 226us) client_negotiation.cc:240] Negotiation successful
0925 11:01:33.260929 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":255,"threads_started":1}
I20260925 11:01:33.263332 1354 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37217
I20260925 11:01:33.263644 1354 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:33.264457 1354 heartbeater.cc:511] Master 127.24.159.254:37217 requested a full tablet report, sending...
I20260925 11:01:33.265928 1031 ts_manager.cc:194] Registered new tserver with Master: 332ce21385d14507aefd003bc596bc33 (127.24.159.194:37187)
I20260925 11:01:33.266755 1031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:37103
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:01:33.296689 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37217
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:33.605774 1359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:33.606069 1359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:33.606228 1359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:33.613358 1359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:33.613469 1359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:33.613619 1359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:33.624137 1359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:33.625375 1359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:33.626688 1359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:33.638900 1364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:33.638957 1365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:33.638988 1367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:33.639707 1359 server_base.cc:1034] running on GCE node
I20260925 11:01:33.640233 1359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:33.641074 1359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:33.642261 1359 hybrid_clock.cc:648] HybridClock initialized: now 1790334093642221 us; error 33 us; skew 500 ppm
I20260925 11:01:33.642545 1359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:33 dist-test-slave-trms krb5kdc[986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334093, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:33.645340 1359 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:01:33.646657 1359 webserver.cc:460] Webserver started at http://127.24.159.195:36865/ using document root <none> and password file <none>
I20260925 11:01:33.647289 1359 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:33.647379 1359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:33.647622 1359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:33.650359 1359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "67acd683e2a441538a1d4fe1301fef3c"
format_stamp: "Formatted at 2026-09-25 11:01:33 on dist-test-slave-trms"
server_key: "a3f964e6d00ec3cc177d1fbba25c8adf"
server_key_iv: "3e4220f939552d7bcd5efac40ba23841"
server_key_version: "encryptionkey@0"
I20260925 11:01:33.651057 1359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "67acd683e2a441538a1d4fe1301fef3c"
format_stamp: "Formatted at 2026-09-25 11:01:33 on dist-test-slave-trms"
server_key: "a3f964e6d00ec3cc177d1fbba25c8adf"
server_key_iv: "3e4220f939552d7bcd5efac40ba23841"
server_key_version: "encryptionkey@0"
I20260925 11:01:33.655766 1359 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 11:01:33.659008 1374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:33.660213 1359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:01:33.660342 1359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "67acd683e2a441538a1d4fe1301fef3c"
format_stamp: "Formatted at 2026-09-25 11:01:33 on dist-test-slave-trms"
server_key: "a3f964e6d00ec3cc177d1fbba25c8adf"
server_key_iv: "3e4220f939552d7bcd5efac40ba23841"
server_key_version: "encryptionkey@0"
I20260925 11:01:33.660516 1359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:33.677923 1359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:33.681645 1359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:33.681896 1359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:33.682816 1359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:33.684051 1359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:33.684120 1359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:33.684231 1359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:33.684279 1359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:33.712209 1359 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:41863
I20260925 11:01:33.712249 1486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:41863 every 8 connection(s)
I20260925 11:01:33.713441 1359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:33.723662 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 1359
I20260925 11:01:33.723814 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:01:33.724225 25215 external_mini_cluster.cc:1428] Setting key 89d34eccfa24e9e63d5735918876a0f5
Sep 25 11:01:33 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334093, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:33.732204 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:33.715838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:59801 (local address 127.24.159.254:37217)
0925 11:01:33.716007 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:33.716011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:33.717216 (+ 1205us) server_negotiation.cc:407] Connection header received
0925 11:01:33.719034 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:33.719040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:33.719120 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:33.719205 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:33.720701 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.721445 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.723069 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.723477 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.723587 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:33.726943 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:33.726947 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:33.726950 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:33.726986 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:33.729277 (+ 2291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:33.729946 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:33.729952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:33.729954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:33.730099 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:33.730520 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:33.730526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:33.730530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:33.730830 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:33.730958 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:33.731654 (+ 696us) server_negotiation.cc:299] Negotiation successful
0925 11:01:33.731867 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 11:01:33.733116 1489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:33.716311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:59801)
0925 11:01:33.717031 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:33.717079 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:33.718762 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:33.719450 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:33.719459 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:33.719966 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:33.720535 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:33.720551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.721586 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.721590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:33.722933 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:33.722944 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.723683 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.723689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:33.723947 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:33.724555 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:33.724591 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:33.726742 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:33.729407 (+ 2665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:33.729421 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:33.729438 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:33.729823 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:33.730212 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:33.730217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:33.730221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:33.730406 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:33.730950 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:33.730958 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:33.731314 (+ 356us) client_negotiation.cc:769] Sending connection context
0925 11:01:33.731548 (+ 234us) client_negotiation.cc:240] Negotiation successful
0925 11:01:33.732059 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":269,"threads_started":1}
I20260925 11:01:33.734423 1487 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37217
I20260925 11:01:33.734731 1487 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:33.735466 1487 heartbeater.cc:511] Master 127.24.159.254:37217 requested a full tablet report, sending...
I20260925 11:01:33.736860 1031 ts_manager.cc:194] Registered new tserver with Master: 67acd683e2a441538a1d4fe1301fef3c (127.24.159.195:41863)
I20260925 11:01:33.737706 1031 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:59801
I20260925 11:01:33.739440 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab.
Entry for principal oryx/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab.
I20260925 11:01:33.793007 1220 heartbeater.cc:503] Master 127.24.159.254:37217 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab.
Entry for principal oryx/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab.
WARNING: no policy specified for oryx/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab.
Entry for principal oryx/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab.
I20260925 11:01:33.860694 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1092
Sep 25 11:01:33 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:01:33.876165 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:33.861772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34662 (local address 127.24.159.193:41593)
0925 11:01:33.862557 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:33.862564 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:33.862588 (+ 24us) server_negotiation.cc:407] Connection header received
0925 11:01:33.862647 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:33.862653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:33.862871 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:33.863011 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:33.864388 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.865443 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.866713 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.867265 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.867373 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:33.871106 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:33.871117 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:33.871131 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:33.871168 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:33.873570 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:33.874205 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:33.874215 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:33.874218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:33.874353 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:33.874694 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:33.874699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:33.874703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:33.875030 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:33.875318 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:33.875578 (+ 260us) server_negotiation.cc:299] Negotiation successful
0925 11:01:33.875835 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":426,"threads_started":1}
I20260925 11:01:33.877379 1215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:33.959419 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 1092
I20260925 11:01:33.972249 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1226
Sep 25 11:01:33 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:01:33.986860 1499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:33.973048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60402 (local address 127.24.159.194:37187)
0925 11:01:33.973521 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:33.973528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:33.973554 (+ 26us) server_negotiation.cc:407] Connection header received
0925 11:01:33.973621 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:33.973628 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:33.973833 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:33.973962 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:33.975118 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.976114 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.977316 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:33.977839 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:33.977935 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:33.981810 (+ 3875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:33.981820 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:33.981833 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:33.981867 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:33.984544 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:33.985125 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:33.985131 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:33.985134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:33.985249 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:33.985515 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:33.985517 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:33.985519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:33.985841 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:33.986067 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:33.986306 (+ 239us) server_negotiation.cc:299] Negotiation successful
0925 11:01:33.986517 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":229,"threads_started":1}
I20260925 11:01:33.988040 1349 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:34.071802 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 1226
I20260925 11:01:34.084743 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1359
Sep 25 11:01:34 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334091, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:01:34.099699 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:34.085599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.24.159.195:41863)
0925 11:01:34.086220 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:34.086231 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:34.086262 (+ 31us) server_negotiation.cc:407] Connection header received
0925 11:01:34.086324 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:34.086331 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:34.086545 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:34.086670 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:34.087884 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:34.088892 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:34.090075 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:34.090746 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:34.090842 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:34.094852 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:34.094864 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:34.094880 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:34.094927 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:34.097479 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:34.098003 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:34.098010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:34.098012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:34.098126 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:34.098437 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:34.098440 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:34.098442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:34.098764 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:34.098996 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:34.099163 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 11:01:34.099348 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":291,"threads_started":1}
I20260925 11:01:34.100925 1482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:34.181726 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 1359
I20260925 11:01:34.195183 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 999
I20260925 11:01:34.196312 1061 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:34.296761 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 999
I20260925 11:01:34.313357 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37217
--webserver_interface=127.24.159.254
--webserver_port=39815
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:37217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:34.628149 1504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:34.628453 1504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:34.628587 1504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:34.635735 1504 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:34.635859 1504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:34.635938 1504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:34.636014 1504 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:34.636077 1504 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:34.646039 1504 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:37217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=39815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:34.647420 1504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:34.648785 1504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:34.660231 1510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:34.660292 1512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:34.660742 1509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:34.660972 1504 server_base.cc:1034] running on GCE node
I20260925 11:01:34.661721 1504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:34.663168 1504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:34.664371 1504 hybrid_clock.cc:648] HybridClock initialized: now 1790334094664318 us; error 40 us; skew 500 ppm
I20260925 11:01:34.664692 1504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:34 dist-test-slave-trms krb5kdc[986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334094, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:34.667569 1504 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:01:34.669008 1504 webserver.cc:460] Webserver started at http://127.24.159.254:39815/ using document root <none> and password file <none>
I20260925 11:01:34.669673 1504 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:34.669795 1504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:34.675138 1504 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:01:34.677811 1519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:34.679078 1504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:34.679224 1504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "60480c443e7e45168e5b312b465a50c1"
format_stamp: "Formatted at 2026-09-25 11:01:32 on dist-test-slave-trms"
server_key: "7990bc5e91042528badb09a8fd9e9594"
server_key_iv: "a74cdfa0baff62fab432a1eff8ac0c8a"
server_key_version: "encryptionkey@0"
I20260925 11:01:34.680158 1504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:34.707259 1504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:34.711174 1504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:34.711421 1504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:34.727614 1504 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:37217
I20260925 11:01:34.727638 1570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:37217 every 8 connection(s)
I20260925 11:01:34.729067 1504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:34.733408 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 1504
I20260925 11:01:34.734134 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:41593
--local_ip_for_outbound_sockets=127.24.159.193
--tserver_master_addrs=127.24.159.254:37217
--webserver_port=35541
--webserver_interface=127.24.159.193
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.159.193
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:01:34.735817 1571 sys_catalog.cc:263] Verifying existing consensus state
I20260925 11:01:34.738955 1571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: Bootstrap starting.
I20260925 11:01:34.764393 1571 log.cc:824] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:34.775674 1571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 11:01:34.776192 1571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: Bootstrap complete.
I20260925 11:01:34.785207 1571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } }
I20260925 11:01:34.786089 1571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60480c443e7e45168e5b312b465a50c1, State: Initialized, Role: FOLLOWER
I20260925 11:01:34.786728 1571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } }
I20260925 11:01:34.786952 1571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:34.787055 1571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:34.787213 1571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 1 FOLLOWER]: Advancing to term 2
I20260925 11:01:34.790921 1571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } }
I20260925 11:01:34.791420 1571 leader_election.cc:302] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60480c443e7e45168e5b312b465a50c1; no voters:
I20260925 11:01:34.791994 1571 leader_election.cc:288] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 11:01:34.792198 1575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 11:01:34.793139 1575 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [term 2 LEADER]: Becoming Leader. State: Replica: 60480c443e7e45168e5b312b465a50c1, State: Running, Role: LEADER
I20260925 11:01:34.793751 1575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } }
I20260925 11:01:34.793901 1571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:34.796481 1577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60480c443e7e45168e5b312b465a50c1. Latest consensus state: current_term: 2 leader_uuid: "60480c443e7e45168e5b312b465a50c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } } }
I20260925 11:01:34.796833 1577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:34.797302 1576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "60480c443e7e45168e5b312b465a50c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60480c443e7e45168e5b312b465a50c1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37217 } } }
I20260925 11:01:34.797590 1576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:34.798935 1580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:34.805145 1580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:34.806672 1580 catalog_manager.cc:1259] Loaded cluster ID: 8eb8b9c55392407b8fde9ba272e06254
I20260925 11:01:34.806849 1580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:34.811075 1580 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:34.813060 1580 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 60480c443e7e45168e5b312b465a50c1: Loaded TSK: 0
I20260925 11:01:34.814822 1580 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 11:01:35.092895 1573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:35.093171 1573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:35.093302 1573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:35.100543 1573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:35.100661 1573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:35.100806 1573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:35.110321 1573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.193.keytab
--principal=oryx/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:41593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=35541
--tserver_master_addrs=127.24.159.254:37217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:35.111572 1573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:35.112812 1573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:35.125233 1597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:35.125435 1600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:35.125761 1598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:35.126430 1573 server_base.cc:1034] running on GCE node
I20260925 11:01:35.127152 1573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:35.128279 1573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:35.129508 1573 hybrid_clock.cc:648] HybridClock initialized: now 1790334095129447 us; error 48 us; skew 500 ppm
I20260925 11:01:35.129870 1573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334095, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:35.133484 1573 init.cc:377] Logged in from keytab as oryx/127.24.159.193@KRBTEST.COM (short username oryx)
I20260925 11:01:35.135092 1573 webserver.cc:460] Webserver started at http://127.24.159.193:35541/ using document root <none> and password file <none>
I20260925 11:01:35.135897 1573 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:35.136035 1573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:35.142627 1573 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 11:01:35.145726 1607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:35.147306 1573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:35.147477 1573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "b422391743734bc6bddbc61ef7a4e9fe"
format_stamp: "Formatted at 2026-09-25 11:01:32 on dist-test-slave-trms"
server_key: "6e54ce28cb118cb3c438d0266ff5c80e"
server_key_iv: "f3596632b8fc04d68aef37c706883692"
server_key_version: "encryptionkey@0"
I20260925 11:01:35.148679 1573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:35.170872 1573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:35.175721 1573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:35.175985 1573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:35.177063 1573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:35.178586 1573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:35.178690 1573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:35.178781 1573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:35.178848 1573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:35.208288 1573 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:41593
I20260925 11:01:35.208352 1719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:41593 every 8 connection(s)
I20260925 11:01:35.209686 1573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:35.218542 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 1573
I20260925 11:01:35.219101 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:37187
--local_ip_for_outbound_sockets=127.24.159.194
--tserver_master_addrs=127.24.159.254:37217
--webserver_port=38907
--webserver_interface=127.24.159.194
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.159.194
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:35.226651 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:35.212444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:59881)
0925 11:01:35.213199 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:35.213247 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:35.215030 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:35.216260 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:35.216267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:35.216757 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:35.217284 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.217301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.218824 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.218828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.220079 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.220091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.221079 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.221086 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.221351 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.221960 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:35.221981 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:35.225331 (+ 3350us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":308,"threads_started":1}
W20260925 11:01:35.226739 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:35.212393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:59881 (local address 127.24.159.254:37217)
0925 11:01:35.213031 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:35.213043 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:35.213351 (+ 308us) server_negotiation.cc:407] Connection header received
0925 11:01:35.215504 (+ 2153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:35.215517 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:35.215832 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:35.216142 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:35.217433 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.218683 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.220217 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.220864 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.221039 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.225709 (+ 4670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:59881: BlockingRecv error: recv got EOF from 127.24.159.193:59881 (error 108)
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":237,"threads_started":1}
W20260925 11:01:35.227533 1720 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:37217 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.159.254:37217: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:35.235209 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:35.228098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:58195)
0925 11:01:35.228254 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:35.228271 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:35.228350 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:35.228711 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:35.228727 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:35.229036 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:35.229166 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.229174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.230150 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.230155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.231063 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.231075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.231695 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.231701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.231869 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.232423 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:35.232439 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:35.234948 (+ 2509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 11:01:35.235357 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:35.228312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:58195 (local address 127.24.159.254:37217)
0925 11:01:35.228482 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:35.228486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:35.228499 (+ 13us) server_negotiation.cc:407] Connection header received
0925 11:01:35.228540 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:35.228544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:35.228596 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:35.228673 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:35.229296 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.230022 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.231192 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.231570 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.231669 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.235151 (+ 3482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:58195: BlockingRecv error: recv got EOF from 127.24.159.193:58195 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:35.244058 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:35.235991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:46171)
0925 11:01:35.236129 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:35.236148 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:35.236235 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:35.236499 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:35.236505 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:35.236785 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:35.236920 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.236928 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.237846 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.237851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.238776 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.238788 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.239836 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.239852 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.240043 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.240576 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:35.240598 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:35.243762 (+ 3164us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 11:01:35.244436 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:35.236098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:46171 (local address 127.24.159.254:37217)
0925 11:01:35.236246 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:35.236251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:35.236268 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:01:35.236315 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:35.236320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:35.236392 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:35.236466 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:35.237044 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.237718 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.238906 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.239501 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.239631 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.244089 (+ 4458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:46171: BlockingRecv error: recv got EOF from 127.24.159.193:46171 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260925 11:01:35.244961 1720 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 11:01:35.541234 1724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:35.541507 1724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:35.541599 1724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:35.548780 1724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:35.548885 1724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:35.549012 1724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:35.558645 1724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.194.keytab
--principal=oryx/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:37187
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=38907
--tserver_master_addrs=127.24.159.254:37217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:35.559861 1724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:35.561092 1724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:35.573426 1730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:35.573449 1732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:35.573508 1729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:35.574115 1724 server_base.cc:1034] running on GCE node
I20260925 11:01:35.574734 1724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:35.575630 1724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:35.576829 1724 hybrid_clock.cc:648] HybridClock initialized: now 1790334095576783 us; error 36 us; skew 500 ppm
I20260925 11:01:35.577126 1724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334095, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:35.580150 1724 init.cc:377] Logged in from keytab as oryx/127.24.159.194@KRBTEST.COM (short username oryx)
I20260925 11:01:35.581507 1724 webserver.cc:460] Webserver started at http://127.24.159.194:38907/ using document root <none> and password file <none>
I20260925 11:01:35.582168 1724 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:35.582332 1724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:35.587656 1724 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:01:35.590539 1739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:35.591836 1724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:35.591992 1724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "332ce21385d14507aefd003bc596bc33"
format_stamp: "Formatted at 2026-09-25 11:01:33 on dist-test-slave-trms"
server_key: "cad96f8496d235ecdc9caad59fb9cb22"
server_key_iv: "dc254083781e059affa33dcf604fc62a"
server_key_version: "encryptionkey@0"
I20260925 11:01:35.592907 1724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:35.606141 1724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:35.610183 1724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:35.610463 1724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:35.611414 1724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:35.612749 1724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:35.612864 1724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:35.612967 1724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:35.613036 1724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:35.643163 1724 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:37187
I20260925 11:01:35.643213 1851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:37187 every 8 connection(s)
I20260925 11:01:35.644536 1724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:35.649338 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 1724
I20260925 11:01:35.649871 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:41863
--local_ip_for_outbound_sockets=127.24.159.195
--tserver_master_addrs=127.24.159.254:37217
--webserver_port=36865
--webserver_interface=127.24.159.195
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.159.195
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 11:01:35.660322 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:35.647031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:52233 (local address 127.24.159.254:37217)
0925 11:01:35.647191 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:35.647196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:35.648295 (+ 1099us) server_negotiation.cc:407] Connection header received
0925 11:01:35.650132 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:35.650137 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:35.650239 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:35.650299 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:35.651993 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.652921 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.654634 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.655043 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.655158 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.660139 (+ 4981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:52233: BlockingRecv error: recv got EOF from 127.24.159.194:52233 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 11:01:35.661234 1854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:35.647497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:52233)
0925 11:01:35.648145 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:35.648181 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:35.649849 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:35.650728 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:35.650736 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:35.651236 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:35.651808 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.651826 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.653055 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.653058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.654484 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.654502 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.655256 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.655263 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.655500 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.656108 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:35.656130 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:35.660000 (+ 3870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":237,"threads_started":1}
W20260925 11:01:35.661867 1852 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:37217 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.159.254:37217: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:35.669970 1854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:35.662435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:43831)
0925 11:01:35.662721 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:35.662735 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:35.662821 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:35.663099 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:35.663102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:35.663371 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:35.663497 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.663504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.664757 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.664760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.665625 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.665636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.666521 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.666525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.666640 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.667138 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:35.667155 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:35.669780 (+ 2625us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 11:01:35.670336 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:35.662521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:43831 (local address 127.24.159.254:37217)
0925 11:01:35.662671 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:35.662675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:35.662795 (+ 120us) server_negotiation.cc:407] Connection header received
0925 11:01:35.662916 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:35.662920 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:35.662992 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:35.663088 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:35.663618 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.664638 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.665755 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.666390 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.666526 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.670057 (+ 3531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:43831: BlockingRecv error: recv got EOF from 127.24.159.194:43831 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:35 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:35.678488 1854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:35.670563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:42269)
0925 11:01:35.670702 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:35.670716 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:35.670800 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:35.671106 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:35.671111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:35.671376 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:35.671508 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.671515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.672533 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.672539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.674014 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:35.674029 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.674712 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.674718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:35.674922 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.675415 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:35.675437 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:35.678284 (+ 2847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 11:01:35.678699 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:35.670701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:42269 (local address 127.24.159.254:37217)
0925 11:01:35.670884 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:35.670890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:35.670905 (+ 15us) server_negotiation.cc:407] Connection header received
0925 11:01:35.670943 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:35.670946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:35.670995 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:35.671070 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:35.671632 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.672382 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.674144 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:35.674585 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:35.674675 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:35.678488 (+ 3813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:42269: BlockingRecv error: recv got EOF from 127.24.159.194:42269 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 11:01:35.678766 1852 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 11:01:35.965399 1855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:35.965662 1855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:35.965783 1855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:35.972872 1855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:35.972973 1855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:35.973121 1855 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:35.982692 1855 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/krb5kdc/oryx_127.24.159.195.keytab
--principal=oryx/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:41863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=36865
--tserver_master_addrs=127.24.159.254:37217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:35.984048 1855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:35.985359 1855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:35.997820 1860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:35.997865 1863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:35.997865 1861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:35.998659 1855 server_base.cc:1034] running on GCE node
I20260925 11:01:35.999223 1855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:36.000157 1855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:36.001351 1855 hybrid_clock.cc:648] HybridClock initialized: now 1790334096001298 us; error 45 us; skew 500 ppm
I20260925 11:01:36.001636 1855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334096, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:36.004599 1855 init.cc:377] Logged in from keytab as oryx/127.24.159.195@KRBTEST.COM (short username oryx)
I20260925 11:01:36.006002 1855 webserver.cc:460] Webserver started at http://127.24.159.195:36865/ using document root <none> and password file <none>
I20260925 11:01:36.006675 1855 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:36.006783 1855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:36.011955 1855 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 11:01:36.014492 1870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:36.015762 1855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:36.015920 1855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "67acd683e2a441538a1d4fe1301fef3c"
format_stamp: "Formatted at 2026-09-25 11:01:33 on dist-test-slave-trms"
server_key: "a3f964e6d00ec3cc177d1fbba25c8adf"
server_key_iv: "3e4220f939552d7bcd5efac40ba23841"
server_key_version: "encryptionkey@0"
I20260925 11:01:36.016803 1855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:36.039873 1855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:36.044005 1855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:36.044250 1855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:36.045203 1855 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:36.046504 1855 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:36.046595 1855 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:36.046698 1855 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:36.046764 1855 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:36.075429 1855 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:41863
I20260925 11:01:36.075500 1982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:41863 every 8 connection(s)
I20260925 11:01:36.076823 1855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:36.081607 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 1855
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 11:01:36.092681 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:36.079321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:38215 (local address 127.24.159.254:37217)
0925 11:01:36.079502 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:36.079507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:36.080663 (+ 1156us) server_negotiation.cc:407] Connection header received
0925 11:01:36.082794 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:36.082801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:36.082917 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:36.082986 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:36.084801 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.085639 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.087292 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.087795 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.087902 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.092397 (+ 4495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:38215: BlockingRecv error: recv got EOF from 127.24.159.195:38215 (error 108)
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":65}
I20260925 11:01:36.093339 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:36.079830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:38215)
0925 11:01:36.080507 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:36.080544 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:36.082445 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:36.083426 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:36.083434 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:36.084030 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:36.084603 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.084627 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.085781 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.085785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.087159 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.087171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.088008 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.088016 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.088219 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.088734 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:36.088755 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:36.092205 (+ 3450us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":248,"threads_started":1}
W20260925 11:01:36.093984 1983 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:37217 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.159.254:37217: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
I20260925 11:01:36.096808 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:36.082516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40636 (local address 127.24.159.254:37217)
0925 11:01:36.082934 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:36.082940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:36.083255 (+ 315us) server_negotiation.cc:407] Connection header received
0925 11:01:36.083419 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:36.083424 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:36.083487 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:36.083578 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:36.085064 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.085871 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.087182 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.087662 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.087768 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.090439 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:36.090448 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:36.090464 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:36.090499 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:36.093319 (+ 2820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:36.094124 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:36.094132 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:36.094139 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:36.094366 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:36.094784 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:36.094788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:36.094792 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:36.095437 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:36.095734 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:36.096113 (+ 379us) server_negotiation.cc:299] Negotiation successful
0925 11:01:36.096486 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":308,"thread_start_us":193,"threads_started":1}
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:36.102358 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:36.094571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:33133)
0925 11:01:36.094728 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:36.094745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:36.094852 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:36.095139 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:36.095144 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:36.095519 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:36.095713 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.095724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.096830 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.096836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.097995 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.098006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.098748 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.098753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.098919 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.099411 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:36.099430 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:36.102081 (+ 2651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 11:01:36.102509 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:36.094603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:33133 (local address 127.24.159.254:37217)
0925 11:01:36.094735 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:36.094741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:36.094814 (+ 73us) server_negotiation.cc:407] Connection header received
0925 11:01:36.094949 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:36.094954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:36.095020 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:36.095086 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:36.095819 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.096708 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.098119 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.098619 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.098724 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.102303 (+ 3579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:33133: BlockingRecv error: recv got EOF from 127.24.159.195:33133 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:36.109611 1985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:36.102987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:59773)
0925 11:01:36.103108 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:36.103121 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:36.103195 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:36.103611 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:36.103616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:36.103871 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:36.104010 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.104018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.104947 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.104953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.105822 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.105832 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.106506 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.106512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.106661 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.107086 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:36.107100 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:36.109414 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260925 11:01:36.109783 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:36.103180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:59773 (local address 127.24.159.254:37217)
0925 11:01:36.103375 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:36.103379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:36.103391 (+ 12us) server_negotiation.cc:407] Connection header received
0925 11:01:36.103429 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:36.103432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:36.103492 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:36.103569 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:36.104134 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.104824 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.105953 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.106381 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.106484 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.109582 (+ 3098us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:59773: BlockingRecv error: recv got EOF from 127.24.159.195:59773 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260925 11:01:36.109885 1983 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:36.253790 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:36.245525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:58507)
0925 11:01:36.246025 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:36.246041 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:36.246156 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:36.246467 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:36.246470 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:36.246893 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:36.247101 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.247110 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.248041 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.248045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.249084 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.249098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.249763 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.249769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.249915 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.250626 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:36.250650 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:36.253494 (+ 2844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":252,"threads_started":1}
W20260925 11:01:36.254145 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:36.245610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:58507 (local address 127.24.159.254:37217)
0925 11:01:36.245779 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:36.245784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:36.246105 (+ 321us) server_negotiation.cc:407] Connection header received
0925 11:01:36.246281 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:36.246285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:36.246345 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:36.246423 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:36.247211 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.247901 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.249193 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.249585 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.249675 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.253756 (+ 4081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:58507: BlockingRecv error: recv got EOF from 127.24.159.193:58507 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:36 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:36.687449 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:36.679338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:49101)
0925 11:01:36.679865 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:36.679880 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:36.680000 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:36.680344 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:36.680348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:36.680844 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:36.681035 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.681044 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.681976 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.681979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.682970 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:36.682982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.683841 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.683846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:36.683969 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.684555 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:36.684572 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:36.687212 (+ 2640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":280,"threads_started":1}
W20260925 11:01:36.687657 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:36.679434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:49101 (local address 127.24.159.254:37217)
0925 11:01:36.679660 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:36.679667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:36.679967 (+ 300us) server_negotiation.cc:407] Connection header received
0925 11:01:36.680111 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:36.680116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:36.680194 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:36.680293 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:36.681155 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.681870 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.683118 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:36.683676 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:36.683799 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:36.687437 (+ 3638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:49101: BlockingRecv error: recv got EOF from 127.24.159.194:49101 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 11:01:37 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:37 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:37.118443 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:37.110486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:45319)
0925 11:01:37.110958 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:37.110973 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:37.111097 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:37.111416 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:37.111421 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:37.111857 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:37.112043 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:37.112052 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.113022 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.113028 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:37.113965 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:37.113976 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.114717 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.114724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:37.114887 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:37.115435 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:37.115452 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:37.118183 (+ 2731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":238,"threads_started":1}
W20260925 11:01:37.118613 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:37.110555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:45319 (local address 127.24.159.254:37217)
0925 11:01:37.110733 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:37.110738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:37.111052 (+ 314us) server_negotiation.cc:407] Connection header received
0925 11:01:37.111209 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:37.111214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:37.111277 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:37.111365 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:37.112179 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.112890 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.114098 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.114555 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.114661 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:37.118411 (+ 3750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:45319: BlockingRecv error: recv got EOF from 127.24.159.195:45319 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 11:01:37 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:37 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:37.262602 1991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:37.254717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:37699)
0925 11:01:37.255289 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:37.255304 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:37.255417 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:37.255723 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:37.255728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:37.256210 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:37.256420 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:37.256434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.257644 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.257647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:37.258611 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:37.258623 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.259252 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.259256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:37.259363 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:37.259865 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:37.259882 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:37.262358 (+ 2476us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":354,"threads_started":1}
W20260925 11:01:37.262748 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:37.254782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:37699 (local address 127.24.159.254:37217)
0925 11:01:37.254946 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:37.254951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:37.255380 (+ 429us) server_negotiation.cc:407] Connection header received
0925 11:01:37.255519 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:37.255523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:37.255588 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:37.255654 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:37.256557 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.257526 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.258729 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.259121 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.259245 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:37.262538 (+ 3293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:37699: BlockingRecv error: recv got EOF from 127.24.159.193:37699 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 11:01:37 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:37 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:37.696419 1992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:37.688306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:50455)
0925 11:01:37.688809 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:37.688824 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:37.688963 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:37.689273 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:37.689278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:37.689673 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:37.689851 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:37.689861 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.690893 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.690899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:37.691792 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:37.691805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.692464 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.692470 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:37.692623 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:37.693134 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:37.693150 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:37.696081 (+ 2931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":264,"threads_started":1}
W20260925 11:01:37.696514 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:37.688386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:50455 (local address 127.24.159.254:37217)
0925 11:01:37.688558 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:37.688564 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:37.688906 (+ 342us) server_negotiation.cc:407] Connection header received
0925 11:01:37.689074 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:37.689077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:37.689152 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:37.689232 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:37.689998 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.690772 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.691924 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:37.692322 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:37.692438 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:37.696292 (+ 3854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:50455: BlockingRecv error: recv got EOF from 127.24.159.194:50455 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 11:01:38 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:38 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:38.129487 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:38.119261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:58021)
0925 11:01:38.119830 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:38.119846 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:38.119967 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:38.120278 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:38.120282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:38.120686 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:38.120890 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:38.120901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.121896 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.121899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:38.122812 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:38.122823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.123427 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.123431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:38.123534 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:38.126683 (+ 3149us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:38.126701 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:38.129234 (+ 2533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":270,"threads_started":1}
W20260925 11:01:38.129662 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:38.119409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:58021 (local address 127.24.159.254:37217)
0925 11:01:38.119579 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:38.119585 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:38.119913 (+ 328us) server_negotiation.cc:407] Connection header received
0925 11:01:38.120056 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:38.120059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:38.120148 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:38.120239 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:38.121028 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.121786 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.122920 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.123311 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.123401 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:38.129418 (+ 6017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:58021: BlockingRecv error: recv got EOF from 127.24.159.195:58021 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 11:01:38 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:38 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:38.271098 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:38.263392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:47599)
0925 11:01:38.263825 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:38.263840 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:38.263963 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:38.264210 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:38.264213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:38.264537 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:38.264710 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:38.264718 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.265800 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.265804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:38.266738 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:38.266750 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.267455 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.267459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:38.267591 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:38.268113 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:38.268129 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:38.270745 (+ 2616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":219,"threads_started":1}
W20260925 11:01:38.271235 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:38.263483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:47599 (local address 127.24.159.254:37217)
0925 11:01:38.263702 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:38.263708 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:38.263909 (+ 201us) server_negotiation.cc:407] Connection header received
0925 11:01:38.264052 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:38.264056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:38.264118 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:38.264194 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:38.264868 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.265647 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.266914 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.267317 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.267418 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:38.270979 (+ 3561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:47599: BlockingRecv error: recv got EOF from 127.24.159.193:47599 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 25 11:01:38 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:38 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:38.704649 1995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:38.697234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:58767)
0925 11:01:38.697676 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:38.697691 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:38.697798 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:38.698057 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:38.698060 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:38.698410 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:38.698577 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:38.698585 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.699552 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.699555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:38.700451 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:38.700462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.701133 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.701136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:38.701244 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:38.701825 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:38.701850 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:38.704303 (+ 2453us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":231,"threads_started":1}
W20260925 11:01:38.704728 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:38.697309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:58767 (local address 127.24.159.254:37217)
0925 11:01:38.697510 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:38.697516 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:38.697759 (+ 243us) server_negotiation.cc:407] Connection header received
0925 11:01:38.697890 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:38.697893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:38.697954 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:38.698027 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:38.698683 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.699450 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.700559 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:38.701033 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:38.701131 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:38.704512 (+ 3381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:58767: BlockingRecv error: recv got EOF from 127.24.159.194:58767 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 25 11:01:39 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:39 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:39.137902 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:39.130434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:48067)
0925 11:01:39.130961 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:39.130977 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:39.131090 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:39.131369 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:39.131373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:39.131693 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:39.131888 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:39.131897 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.132820 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.132824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:39.133708 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:39.133746 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.134399 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.134404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:39.134540 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:39.135139 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:39.135156 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:39.137639 (+ 2483us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":293,"threads_started":1}
W20260925 11:01:39.138019 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:39.130550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:48067 (local address 127.24.159.254:37217)
0925 11:01:39.130704 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:39.130709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:39.131046 (+ 337us) server_negotiation.cc:407] Connection header received
0925 11:01:39.131184 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:39.131187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:39.131257 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:39.131336 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:39.132004 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.132695 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.133843 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.134251 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.134336 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:39.137831 (+ 3495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:48067: BlockingRecv error: recv got EOF from 127.24.159.195:48067 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 25 11:01:39 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:39 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:39.279781 1997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:39.272066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:51349)
0925 11:01:39.272543 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:39.272559 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:39.272669 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:39.272935 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:39.272937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:39.273287 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:39.273448 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:39.273456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.274398 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.274401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:39.275389 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:39.275402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.275993 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.275997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:39.276113 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:39.276622 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:39.276639 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:39.279447 (+ 2808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":247,"threads_started":1}
W20260925 11:01:39.279946 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:39.272156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:51349 (local address 127.24.159.254:37217)
0925 11:01:39.272332 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:39.272337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:39.272621 (+ 284us) server_negotiation.cc:407] Connection header received
0925 11:01:39.272770 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:39.272774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:39.272836 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:39.272914 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:39.273555 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.274303 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.275503 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.275886 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.275970 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:39.279703 (+ 3733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:51349: BlockingRecv error: recv got EOF from 127.24.159.193:51349 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 25 11:01:39 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:39 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:39.713238 1998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:39.705580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:36515)
0925 11:01:39.706058 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:39.706077 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:39.706240 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:39.706622 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:39.706627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:39.707104 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:39.707274 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:39.707283 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.708248 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.708253 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:39.709244 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:39.709255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.709873 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.709877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:39.710016 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:39.710533 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:39.710550 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:39.713017 (+ 2467us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":253,"threads_started":1}
W20260925 11:01:39.713394 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:39.705642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:36515 (local address 127.24.159.254:37217)
0925 11:01:39.705841 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:39.705848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:39.706160 (+ 312us) server_negotiation.cc:407] Connection header received
0925 11:01:39.706364 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:39.706368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:39.706448 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:39.706548 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:39.707403 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.708121 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.709346 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:39.709740 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:39.709823 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:39.713186 (+ 3363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:36515: BlockingRecv error: recv got EOF from 127.24.159.194:36515 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 11:01:40 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:40 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:40.146430 1999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:40.138756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:58103)
0925 11:01:40.139171 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:40.139187 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:40.139296 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:40.139571 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:40.139574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:40.139935 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:40.140086 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:40.140094 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.141036 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.141040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:40.142062 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:40.142075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.142924 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.142928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:40.143067 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:40.143645 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:40.143661 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:40.146166 (+ 2505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":220,"threads_started":1}
W20260925 11:01:40.146592 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:40.138812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:58103 (local address 127.24.159.254:37217)
0925 11:01:40.139012 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:40.139019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:40.139257 (+ 238us) server_negotiation.cc:407] Connection header received
0925 11:01:40.139382 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:40.139386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:40.139468 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:40.139547 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:40.140193 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.140939 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.142181 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.142787 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.142901 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:40.146385 (+ 3484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:58103: BlockingRecv error: recv got EOF from 127.24.159.195:58103 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 25 11:01:40 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:40 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:40.288552 2000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:40.280754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:51037)
0925 11:01:40.281250 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:40.281266 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:40.281398 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:40.281707 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:40.281712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:40.282122 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:40.282330 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:40.282341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.283338 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.283344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:40.284345 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:40.284359 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.285034 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.285040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:40.285200 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:40.285707 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:40.285724 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:40.288229 (+ 2505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":241,"threads_started":1}
W20260925 11:01:40.288648 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:40.280827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:51037 (local address 127.24.159.254:37217)
0925 11:01:40.281042 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:40.281049 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:40.281364 (+ 315us) server_negotiation.cc:407] Connection header received
0925 11:01:40.281513 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:40.281517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:40.281577 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:40.281664 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:40.282480 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.283209 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.284472 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.284900 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.285005 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:40.288452 (+ 3447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:51037: BlockingRecv error: recv got EOF from 127.24.159.193:51037 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 25 11:01:40 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:40 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:40.722288 2001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:40.714140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:53547)
0925 11:01:40.714733 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:40.714749 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:40.714877 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:40.715189 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:40.715194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:40.715610 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:40.715761 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:40.715769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.716875 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.716881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:40.717804 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:40.717817 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.718571 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.718577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:40.718737 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:40.719268 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:40.719284 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:40.721930 (+ 2646us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":295,"threads_started":1}
W20260925 11:01:40.722397 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:40.714369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:53547 (local address 127.24.159.254:37217)
0925 11:01:40.714564 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:40.714570 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:40.714836 (+ 266us) server_negotiation.cc:407] Connection header received
0925 11:01:40.714992 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:40.714998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:40.715067 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:40.715146 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:40.715908 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.716737 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.717945 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:40.718438 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:40.718544 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:40.722132 (+ 3588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:53547: BlockingRecv error: recv got EOF from 127.24.159.194:53547 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 25 11:01:41 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:41 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
W20260925 11:01:41.155212 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:41.147367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:48313 (local address 127.24.159.254:37217)
0925 11:01:41.147524 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:41.147529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:41.147886 (+ 357us) server_negotiation.cc:407] Connection header received
0925 11:01:41.148018 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:41.148021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:41.148079 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:41.148155 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:41.148862 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.149541 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.150752 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.151137 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.151221 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:41.154949 (+ 3728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:48313: BlockingRecv error: recv got EOF from 127.24.159.195:48313 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 11:01:41.155673 2002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:41.147263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:48313)
0925 11:01:41.147808 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:41.147824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:41.147931 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:41.148181 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:41.148184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:41.148532 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:41.148692 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:41.148701 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.149681 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.149686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:41.150641 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:41.150653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.151252 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.151257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:41.151378 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:41.151915 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:41.151932 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:41.155441 (+ 3509us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":281,"threads_started":1}
Sep 25 11:01:41 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:41 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:41.297433 2003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:41.289430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:60217)
0925 11:01:41.289930 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:41.289945 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:41.290073 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:41.290402 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:41.290406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:41.290804 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:41.290969 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:41.290978 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.291978 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.291982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:41.292856 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:41.292868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.293538 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.293544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:41.293712 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:41.294309 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:41.294332 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:41.297190 (+ 2858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":274,"threads_started":1}
W20260925 11:01:41.297631 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:41.289521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:60217 (local address 127.24.159.254:37217)
0925 11:01:41.289706 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:41.289712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:41.290030 (+ 318us) server_negotiation.cc:407] Connection header received
0925 11:01:41.290206 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:41.290211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:41.290274 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:41.290352 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:41.291107 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.291844 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.292997 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.293386 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.293499 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:41.297406 (+ 3907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:60217: BlockingRecv error: recv got EOF from 127.24.159.193:60217 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 25 11:01:41 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:41 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:41.731017 2004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:41.723141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:58391)
0925 11:01:41.723596 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:41.723613 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:41.723724 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:41.723979 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:41.723982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:41.724331 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:41.724495 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:41.724504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.725389 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.725392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:41.726292 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:41.726305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.726906 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.726910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:41.727024 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:41.727672 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:41.727694 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:41.730802 (+ 3108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":243,"threads_started":1}
W20260925 11:01:41.731547 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:41.723240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:58391 (local address 127.24.159.254:37217)
0925 11:01:41.723455 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:41.723461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:41.723697 (+ 236us) server_negotiation.cc:407] Connection header received
0925 11:01:41.723807 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:41.723811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:41.723875 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:41.723950 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:41.724600 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.725290 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.726400 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:41.726796 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:41.726887 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:41.731075 (+ 4188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:58391: BlockingRecv error: recv got EOF from 127.24.159.194:58391 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Sep 25 11:01:42 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:42 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:42.164067 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:42.156511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:47061)
0925 11:01:42.156955 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:42.156970 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:42.157080 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:42.157336 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:42.157339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:42.157702 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:42.157848 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:42.157857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.158815 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.158819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:42.159704 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:42.159715 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.160468 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.160473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:42.160605 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:42.161167 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:42.161183 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:42.163833 (+ 2650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":237,"threads_started":1}
W20260925 11:01:42.164321 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:42.156587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:47061 (local address 127.24.159.254:37217)
0925 11:01:42.156805 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:42.156812 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:42.157044 (+ 232us) server_negotiation.cc:407] Connection header received
0925 11:01:42.157169 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:42.157172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:42.157233 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:42.157308 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:42.157951 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.158718 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.159851 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.160300 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.160404 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:42.164066 (+ 3662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:47061: BlockingRecv error: recv got EOF from 127.24.159.195:47061 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Sep 25 11:01:42 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:42 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:42.305680 2006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:42.298248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:39083)
0925 11:01:42.298828 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:42.298844 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:42.298967 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:42.299216 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:42.299218 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:42.299636 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:42.299814 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:42.299823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.300789 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.300793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:42.301708 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:42.301719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.302362 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.302367 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:42.302475 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:42.302958 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:42.302974 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:42.305392 (+ 2418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":253,"threads_started":1}
W20260925 11:01:42.305825 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:42.298325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:39083 (local address 127.24.159.254:37217)
0925 11:01:42.298511 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:42.298515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:42.298917 (+ 402us) server_negotiation.cc:407] Connection header received
0925 11:01:42.299045 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:42.299049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:42.299110 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:42.299187 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:42.299952 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.300677 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.301827 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.302251 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.302340 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:42.305625 (+ 3285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:39083: BlockingRecv error: recv got EOF from 127.24.159.193:39083 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 11:01:42 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:42 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:42.740249 2007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:42.732042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:44777)
0925 11:01:42.732590 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:42.732610 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:42.732743 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:42.733119 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:42.733124 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:42.733804 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:42.734014 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:42.734025 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.735147 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.735150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:42.736056 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:42.736068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.736757 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.736761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:42.736878 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:42.737417 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:42.737434 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:42.740003 (+ 2569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":240,"threads_started":1}
W20260925 11:01:42.740466 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:42.732139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:44777 (local address 127.24.159.254:37217)
0925 11:01:42.732391 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:42.732398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:42.732689 (+ 291us) server_negotiation.cc:407] Connection header received
0925 11:01:42.732889 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:42.732894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:42.732974 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:42.733063 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:42.734164 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.735027 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.736203 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:42.736640 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:42.736741 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:42.740223 (+ 3482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:44777: BlockingRecv error: recv got EOF from 127.24.159.194:44777 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 25 11:01:43 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:43 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:43.180902 2008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:43.165187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:33453)
0925 11:01:43.165761 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:43.165777 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:43.165908 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:43.166235 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:43.166240 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:43.166654 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:43.166832 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:43.166841 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.167817 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.167821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:43.168747 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:43.168759 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.169453 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.169458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:43.169613 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:43.177739 (+ 8126us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:43.177761 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:43.180628 (+ 2867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":262,"threads_started":1}
W20260925 11:01:43.181082 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:43.165314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:33453 (local address 127.24.159.254:37217)
0925 11:01:43.165530 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:43.165536 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:43.165861 (+ 325us) server_negotiation.cc:407] Connection header received
0925 11:01:43.166021 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:43.166025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:43.166085 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:43.166166 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:43.166971 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.167683 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.168890 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.169304 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.169412 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:43.180891 (+ 11479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:33453: BlockingRecv error: recv got EOF from 127.24.159.195:33453 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Sep 25 11:01:43 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:43 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:43.314240 2009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:43.306703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:60127)
0925 11:01:43.307184 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:43.307200 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:43.307310 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:43.307562 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:43.307565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:43.308038 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:43.308234 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:43.308246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.309204 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.309208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:43.310163 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:43.310175 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.310894 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.310898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:43.311007 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:43.311530 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:43.311547 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:43.313957 (+ 2410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":231,"threads_started":1}
W20260925 11:01:43.314498 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:43.306855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:60127 (local address 127.24.159.254:37217)
0925 11:01:43.307127 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:43.307132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:43.307263 (+ 131us) server_negotiation.cc:407] Connection header received
0925 11:01:43.307394 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:43.307398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:43.307459 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:43.307531 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:43.308359 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.309076 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.310309 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.310750 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.310840 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:43.314233 (+ 3393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:60127: BlockingRecv error: recv got EOF from 127.24.159.193:60127 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 25 11:01:43 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:43 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:43.748872 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:43.741027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:53735)
0925 11:01:43.741520 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:43.741535 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:43.741662 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:43.741970 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:43.741975 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:43.742395 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:43.742553 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:43.742563 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.743708 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.743712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:43.744622 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:43.744634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.745341 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.745347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:43.745493 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:43.746009 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:43.746025 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:43.748574 (+ 2549us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":240,"threads_started":1}
W20260925 11:01:43.748940 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:43.741124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:53735 (local address 127.24.159.254:37217)
0925 11:01:43.741334 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:43.741341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:43.741610 (+ 269us) server_negotiation.cc:407] Connection header received
0925 11:01:43.741768 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:43.741774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:43.741836 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:43.741921 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:43.742679 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.743573 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.744748 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:43.745178 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:43.745280 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:43.748759 (+ 3479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:53735: BlockingRecv error: recv got EOF from 127.24.159.194:53735 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Sep 25 11:01:44 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:44 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:44.189881 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:44.181806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:38659)
0925 11:01:44.182360 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:44.182376 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:44.182512 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:44.182810 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:44.182815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:44.183221 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:44.183368 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:44.183379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.184624 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.184629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:44.185586 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:44.185598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.186293 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.186298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:44.186423 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:44.186961 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:44.186982 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:44.189590 (+ 2608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":275,"threads_started":1}
W20260925 11:01:44.190042 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:44.181887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:38659 (local address 127.24.159.254:37217)
0925 11:01:44.182047 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:44.182051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:44.182473 (+ 422us) server_negotiation.cc:407] Connection header received
0925 11:01:44.182622 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:44.182626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:44.182687 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:44.182764 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:44.183558 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.184484 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.185705 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.186123 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.186253 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:44.189805 (+ 3552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:38659: BlockingRecv error: recv got EOF from 127.24.159.195:38659 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 25 11:01:44 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:44 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:44.323060 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:44.315297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:50045)
0925 11:01:44.315789 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:44.315805 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:44.315918 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:44.316169 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:44.316172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:44.316545 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:44.316711 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:44.316720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.317696 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.317700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:44.318686 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:44.318699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.319355 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.319360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:44.319481 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:44.320046 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:44.320064 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:44.322816 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":264,"threads_started":1}
W20260925 11:01:44.323202 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:44.315343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:50045 (local address 127.24.159.254:37217)
0925 11:01:44.315523 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:44.315528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:44.315868 (+ 340us) server_negotiation.cc:407] Connection header received
0925 11:01:44.316002 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:44.316005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:44.316064 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:44.316139 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:44.316842 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.317556 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.318799 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.319218 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.319304 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:44.323005 (+ 3701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:50045: BlockingRecv error: recv got EOF from 127.24.159.193:50045 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 11:01:44 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:44 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:44.757898 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:44.749785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:46037)
0925 11:01:44.750301 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:44.750317 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:44.750442 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:44.750691 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:44.750694 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:44.751072 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:44.751231 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:44.751240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.752422 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.752427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:44.753695 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:44.753709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.754374 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.754379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:44.754517 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:44.755030 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:44.755046 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:44.757668 (+ 2622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":277,"threads_started":1}
W20260925 11:01:44.758038 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:44.749893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:46037 (local address 127.24.159.254:37217)
0925 11:01:44.750048 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:44.750052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:44.750386 (+ 334us) server_negotiation.cc:407] Connection header received
0925 11:01:44.750524 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:44.750527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:44.750587 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:44.750666 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:44.751350 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.752278 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.753825 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:44.754257 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:44.754341 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:44.757841 (+ 3500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:46037: BlockingRecv error: recv got EOF from 127.24.159.194:46037 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 11:01:45 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:45 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:45.203009 2014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:45.190882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:52455)
0925 11:01:45.191375 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:45.191391 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:45.191507 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:45.191772 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:45.191775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:45.192173 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:45.192352 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:45.192363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.193279 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.193282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:45.194239 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:45.194255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.194979 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.194984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:45.195109 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:45.199333 (+ 4224us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:45.199351 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:45.202765 (+ 3414us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":270,"threads_started":1}
W20260925 11:01:45.203115 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:45.190924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:52455 (local address 127.24.159.254:37217)
0925 11:01:45.191083 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:45.191088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:45.191455 (+ 367us) server_negotiation.cc:407] Connection header received
0925 11:01:45.191605 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:45.191609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:45.191670 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:45.191742 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:45.192474 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.193178 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.194420 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.194837 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.194955 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:45.202941 (+ 7986us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:52455: BlockingRecv error: recv got EOF from 127.24.159.195:52455 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 11:01:45 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:45 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:45.336721 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:45.324104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:40269)
0925 11:01:45.327683 (+ 3579us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:45.327701 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:45.327838 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:45.328192 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:45.328197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:45.328669 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:45.328866 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:45.328877 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.330322 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.330328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:45.331861 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:45.331877 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.332857 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.332861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:45.332974 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:45.333615 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:45.333641 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:45.336500 (+ 2859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3461,"thread_start_us":3349,"threads_started":1}
W20260925 11:01:45.336894 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:45.324116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:40269 (local address 127.24.159.254:37217)
0925 11:01:45.324295 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:45.324301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:45.327797 (+ 3496us) server_negotiation.cc:407] Connection header received
0925 11:01:45.327966 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:45.327971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:45.328063 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:45.328171 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:45.329018 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.330175 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.332037 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.332717 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.332847 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:45.336685 (+ 3838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:40269: BlockingRecv error: recv got EOF from 127.24.159.193:40269 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 25 11:01:45 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:45 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:45.766052 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:45.758743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:47081)
0925 11:01:45.759213 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:45.759230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:45.759336 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:45.759602 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:45.759604 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:45.759954 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:45.760095 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:45.760103 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.761037 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.761041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:45.761976 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:45.761986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.762656 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.762660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:45.762766 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:45.763296 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:45.763313 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:45.765820 (+ 2507us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":266,"threads_started":1}
W20260925 11:01:45.766278 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:45.758820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:47081 (local address 127.24.159.254:37217)
0925 11:01:45.758980 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:45.758984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:45.759292 (+ 308us) server_negotiation.cc:407] Connection header received
0925 11:01:45.759428 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:45.759432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:45.759493 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:45.759570 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:45.760205 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.760906 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.762101 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:45.762538 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:45.762632 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:45.766010 (+ 3378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:47081: BlockingRecv error: recv got EOF from 127.24.159.194:47081 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Sep 25 11:01:46 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:46 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:46.211516 2017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:46.203957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:38947)
0925 11:01:46.204453 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:46.204469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:46.204572 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:46.204802 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:46.204805 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:46.205160 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:46.205301 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:46.205309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.206236 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.206239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:46.207154 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:46.207165 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.207766 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.207769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:46.207875 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:46.208379 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:46.208395 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:46.211242 (+ 2847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":268,"threads_started":1}
W20260925 11:01:46.211596 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:46.204065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:38947 (local address 127.24.159.254:37217)
0925 11:01:46.204221 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:46.204225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:46.204531 (+ 306us) server_negotiation.cc:407] Connection header received
0925 11:01:46.204651 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:46.204654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:46.204713 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:46.204788 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:46.205413 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.206127 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.207254 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.207660 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.207746 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:46.211418 (+ 3672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:38947: BlockingRecv error: recv got EOF from 127.24.159.195:38947 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 25 11:01:46 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:46 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:46.345511 2018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:46.337592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:55703)
0925 11:01:46.338096 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:46.338111 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:46.338258 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:46.338590 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:46.338594 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:46.339001 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:46.339167 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:46.339176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.340170 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.340174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:46.341048 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:46.341059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.341727 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.341732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:46.341894 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:46.342510 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:46.342528 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:46.345194 (+ 2666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":282,"threads_started":1}
W20260925 11:01:46.345620 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:46.337690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:55703 (local address 127.24.159.254:37217)
0925 11:01:46.337924 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:46.337931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:46.338210 (+ 279us) server_negotiation.cc:407] Connection header received
0925 11:01:46.338376 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:46.338382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:46.338459 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:46.338539 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:46.339302 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.340042 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.341179 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.341586 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.341691 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:46.345420 (+ 3729us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:55703: BlockingRecv error: recv got EOF from 127.24.159.193:55703 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 11:01:46 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:46 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:46.775112 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:46.766959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:40325)
0925 11:01:46.767449 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:46.767466 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:46.767595 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:46.767921 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:46.767926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:46.768328 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:46.768504 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:46.768513 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.769514 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.769518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:46.770648 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:46.770664 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.771427 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.771432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:46.771555 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:46.772151 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:46.772168 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:46.774799 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":262,"threads_started":1}
W20260925 11:01:46.775249 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:46.767075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:40325 (local address 127.24.159.254:37217)
0925 11:01:46.767255 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:46.767261 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:46.767541 (+ 280us) server_negotiation.cc:407] Connection header received
0925 11:01:46.767706 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:46.767712 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:46.767782 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:46.767860 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:46.768640 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.769389 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.770775 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:46.771292 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:46.771389 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:46.775039 (+ 3650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:40325: BlockingRecv error: recv got EOF from 127.24.159.194:40325 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 25 11:01:47 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:47 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:47.220284 2020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:47.212383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:44289)
0925 11:01:47.212824 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:47.212840 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:47.212948 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:47.213204 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:47.213206 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:47.213580 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:47.213755 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:47.213764 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.214742 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.214745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:47.215633 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:47.215644 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.216323 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.216327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:47.216461 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:47.216985 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:47.217000 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:47.220034 (+ 3034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":225,"threads_started":1}
W20260925 11:01:47.220379 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:47.212469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:44289 (local address 127.24.159.254:37217)
0925 11:01:47.212624 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:47.212629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:47.212903 (+ 274us) server_negotiation.cc:407] Connection header received
0925 11:01:47.213034 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:47.213037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:47.213100 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:47.213179 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:47.213887 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.214647 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.215732 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.216143 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.216255 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:47.220197 (+ 3942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:44289: BlockingRecv error: recv got EOF from 127.24.159.195:44289 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 11:01:47 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:47 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:47.354403 2021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:47.346396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:54831)
0925 11:01:47.346874 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:47.346893 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:47.347037 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:47.347425 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:47.347430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:47.347880 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:47.348032 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:47.348044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.349184 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.349188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:47.350092 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:47.350103 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.350806 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.350811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:47.350945 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:47.351479 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:47.351497 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:47.354094 (+ 2597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":260,"threads_started":1}
W20260925 11:01:47.354529 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:47.346479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:54831 (local address 127.24.159.254:37217)
0925 11:01:47.346655 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:47.346661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:47.346986 (+ 325us) server_negotiation.cc:407] Connection header received
0925 11:01:47.347174 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:47.347179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:47.347278 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:47.347358 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:47.348182 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.349060 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.350229 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.350684 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.350778 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:47.354308 (+ 3530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:54831: BlockingRecv error: recv got EOF from 127.24.159.193:54831 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 11:01:47 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:47 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:47.783826 2022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:47.776063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:54051)
0925 11:01:47.776533 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:47.776548 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:47.776675 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:47.776996 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:47.777002 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:47.777394 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:47.777542 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:47.777551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.778576 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.778581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:47.779489 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:47.779502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.780168 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.780173 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:47.780335 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:47.780912 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:47.780937 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:47.783588 (+ 2651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":268,"threads_started":1}
W20260925 11:01:47.784001 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:47.776122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:54051 (local address 127.24.159.254:37217)
0925 11:01:47.776320 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:47.776327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:47.776628 (+ 301us) server_negotiation.cc:407] Connection header received
0925 11:01:47.776782 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:47.776788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:47.776856 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:47.776937 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:47.777678 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.778448 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.779628 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:47.780037 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:47.780141 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:47.783796 (+ 3655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:54051: BlockingRecv error: recv got EOF from 127.24.159.194:54051 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 11:01:48 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:48 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:48.231669 2023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:48.221103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:58697)
0925 11:01:48.221583 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:48.221598 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:48.221709 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:48.221970 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:48.221973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:48.222368 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:48.222550 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:48.222562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.223853 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.223858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:48.224757 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:48.224769 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.225363 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.225367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:48.225497 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:48.228898 (+ 3401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:48.228916 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:48.231440 (+ 2524us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":273,"threads_started":1}
W20260925 11:01:48.231853 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:48.221139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:58697 (local address 127.24.159.254:37217)
0925 11:01:48.221312 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:48.221317 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:48.221662 (+ 345us) server_negotiation.cc:407] Connection header received
0925 11:01:48.221795 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:48.221798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:48.221861 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:48.221944 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:48.222685 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.223719 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.224877 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.225263 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.225350 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:48.231644 (+ 6294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:58697: BlockingRecv error: recv got EOF from 127.24.159.195:58697 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 25 11:01:48 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:48 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:48.362845 2024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:48.355224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:37519)
0925 11:01:48.355675 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:48.355690 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:48.355799 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:48.356068 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:48.356070 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:48.356432 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:48.356589 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:48.356598 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.357596 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.357601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:48.358586 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:48.358599 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.359286 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.359291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:48.359423 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:48.360001 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:48.360018 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:48.362615 (+ 2597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":217,"threads_started":1}
W20260925 11:01:48.363003 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:48.355301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:37519 (local address 127.24.159.254:37217)
0925 11:01:48.355500 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:48.355506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:48.355765 (+ 259us) server_negotiation.cc:407] Connection header received
0925 11:01:48.355884 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:48.355887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:48.355960 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:48.356037 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:48.356723 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.357449 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.358697 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.359116 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.359207 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:48.362797 (+ 3590us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:37519: BlockingRecv error: recv got EOF from 127.24.159.193:37519 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 25 11:01:48 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:48 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:48.792548 2025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:48.784742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:40291)
0925 11:01:48.785211 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:48.785227 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:48.785337 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:48.785621 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:48.785624 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:48.785980 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:48.786146 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:48.786158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.787322 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.787327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:48.788230 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:48.788243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.788946 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.788952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:48.789100 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:48.789632 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:48.789649 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:48.792297 (+ 2648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":243,"threads_started":1}
W20260925 11:01:48.792662 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:48.784884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:40291 (local address 127.24.159.254:37217)
0925 11:01:48.785126 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:48.785133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:48.785296 (+ 163us) server_negotiation.cc:407] Connection header received
0925 11:01:48.785427 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:48.785430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:48.785510 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:48.785589 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:48.786312 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.787198 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.788369 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:48.788782 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:48.788892 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:48.792479 (+ 3587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:40291: BlockingRecv error: recv got EOF from 127.24.159.194:40291 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Sep 25 11:01:49 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:49 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:49.240445 2026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:49.232550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:51671)
0925 11:01:49.233014 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:49.233030 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:49.233142 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:49.233403 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:49.233406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:49.233762 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:49.233947 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:49.233955 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.234972 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.234976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:49.235880 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:49.235891 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.236617 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.236622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:49.236737 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:49.237403 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:49.237421 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:49.240181 (+ 2760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":228,"threads_started":1}
W20260925 11:01:49.240595 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:49.232555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:51671 (local address 127.24.159.254:37217)
0925 11:01:49.232732 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:49.232738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:49.233103 (+ 365us) server_negotiation.cc:407] Connection header received
0925 11:01:49.233242 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:49.233246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:49.233308 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:49.233376 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:49.234092 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.234866 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.236043 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.236477 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.236579 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:49.240405 (+ 3826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:51671: BlockingRecv error: recv got EOF from 127.24.159.195:51671 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 25 11:01:49 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:49 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:49.371068 2027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:49.363670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:53495)
0925 11:01:49.364113 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:49.364129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:49.364240 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:49.364520 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:49.364523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:49.364877 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:49.365054 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:49.365063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.366001 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.366004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:49.366924 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:49.366935 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.367520 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.367524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:49.367642 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:49.368226 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:49.368243 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:49.370832 (+ 2589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":224,"threads_started":1}
W20260925 11:01:49.371227 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:49.363731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:53495 (local address 127.24.159.254:37217)
0925 11:01:49.363980 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:49.363984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:49.364194 (+ 210us) server_negotiation.cc:407] Connection header received
0925 11:01:49.364329 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:49.364333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:49.364408 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:49.364487 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:49.365158 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.365895 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.367024 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.367401 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.367496 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:49.371006 (+ 3510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:53495: BlockingRecv error: recv got EOF from 127.24.159.193:53495 (error 108)
Metrics: {"server-negotiator.queue_time_us":144}
Sep 25 11:01:49 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:49 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:49.801613 2028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:49.793455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:51349)
0925 11:01:49.793927 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:49.793949 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:49.794067 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:49.794408 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:49.794411 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:49.794746 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:49.794930 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:49.794939 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.795981 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.795985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:49.797211 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:49.797227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.797939 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.797945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:49.798110 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:49.798660 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:49.798677 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:49.801270 (+ 2593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":223,"threads_started":1}
W20260925 11:01:49.801704 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:49.793563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:51349 (local address 127.24.159.254:37217)
0925 11:01:49.793735 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:49.793740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:49.794018 (+ 278us) server_negotiation.cc:407] Connection header received
0925 11:01:49.794211 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:49.794217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:49.794300 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:49.794383 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:49.795060 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.795810 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.797342 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:49.797797 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:49.797905 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:49.801485 (+ 3580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:51349: BlockingRecv error: recv got EOF from 127.24.159.194:51349 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 11:01:50 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:50 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.195@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:50.249506 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:50.241408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.195:34967)
0925 11:01:50.241905 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:50.241921 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:50.242049 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:50.242374 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:50.242379 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:50.242739 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:50.242914 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:50.242924 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.243891 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.243897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:50.245035 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:50.245046 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.245922 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.245927 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:50.246046 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:50.246601 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:50.246616 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:50.249184 (+ 2568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":240,"threads_started":1}
W20260925 11:01:50.249642 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:50.241527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:34967 (local address 127.24.159.254:37217)
0925 11:01:50.241746 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:50.241752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:50.242000 (+ 248us) server_negotiation.cc:407] Connection header received
0925 11:01:50.242165 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:50.242168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:50.242250 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:50.242328 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:50.243052 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.243751 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.245163 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.245768 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.245854 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:50.249429 (+ 3575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.195:34967: BlockingRecv error: recv got EOF from 127.24.159.195:34967 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 11:01:50 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:50 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.193@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:50.380290 2030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:50.372094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.193:35623)
0925 11:01:50.372583 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:50.372599 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:50.372709 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:50.373024 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:50.373029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:50.373503 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:50.373726 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:50.373742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.375026 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.375030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:50.375966 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:50.375979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.376755 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.376759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:50.376900 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:50.377438 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:50.377454 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:50.380043 (+ 2589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":268,"threads_started":1}
W20260925 11:01:50.380455 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:50.372180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:35623 (local address 127.24.159.254:37217)
0925 11:01:50.372354 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:50.372358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:50.372666 (+ 308us) server_negotiation.cc:407] Connection header received
0925 11:01:50.372801 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:50.372805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:50.372882 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:50.372966 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:50.373886 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.374865 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.376123 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.376603 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.376706 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:50.380232 (+ 3526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.193:35623: BlockingRecv error: recv got EOF from 127.24.159.193:35623 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Sep 25 11:01:50 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
Sep 25 11:01:50 dist-test-slave-trms krb5kdc[986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.159.194@KRBTEST.COM for oryx/127.24.159.254@KRBTEST.COM, Server not found in Kerberos database
I20260925 11:01:50.810398 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:50.802720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37217 (local address 127.24.159.194:44323)
0925 11:01:50.803181 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:50.803196 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:50.803317 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:50.803611 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:50.803615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:50.803998 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:50.804154 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:50.804163 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.805152 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.805156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:50.806289 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:50.806303 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.807011 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.807015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:50.807137 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:50.807648 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:50.807664 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:50.810109 (+ 2445us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.159.254:37217: Server oryx/127.24.159.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":252,"threads_started":1}
W20260925 11:01:50.810545 1722 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:50.802806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:44323 (local address 127.24.159.254:37217)
0925 11:01:50.802996 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:50.803003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:50.803272 (+ 269us) server_negotiation.cc:407] Connection header received
0925 11:01:50.803425 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:50.803430 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:50.803491 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:50.803569 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:50.804287 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.805002 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.806407 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:50.806885 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:50.806987 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:50.810322 (+ 3335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.159.194:44323: BlockingRecv error: recv got EOF from 127.24.159.194:44323 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 11:01:51.086973 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1573
I20260925 11:01:51.098019 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:51.087883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44120 (local address 127.24.159.193:41593)
0925 11:01:51.088364 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:51.088372 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:51.088400 (+ 28us) server_negotiation.cc:407] Connection header received
0925 11:01:51.088463 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:51.088469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:51.088680 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:51.088861 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:51.090156 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:51.091195 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:51.092289 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:51.092833 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:51.092942 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:51.095255 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:51.095264 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:51.095277 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:51.095312 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:51.097516 (+ 2204us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.159.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 11:01:51.097779 (+ 263us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44120: Request ticket server kudu/127.24.159.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":227,"threads_started":1}
W20260925 11:01:51.098112 2033 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44120: Request ticket server kudu/127.24.159.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 11:01:51.098132 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1573: Not authorized: Client connection negotiation failed: client connection to 127.24.159.193:41593: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.159.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 11:01:51.098359 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 1573
I20260925 11:01:51.112041 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1724
I20260925 11:01:51.123447 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:51.113073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.24.159.194:37187)
0925 11:01:51.113614 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:51.113621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:51.113650 (+ 29us) server_negotiation.cc:407] Connection header received
0925 11:01:51.113708 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:51.113714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:51.113933 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:51.114114 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:51.115385 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:51.116403 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:51.117575 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:51.118116 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:51.118325 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:51.120608 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:51.120617 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:51.120631 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:51.120663 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:51.122844 (+ 2181us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.159.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 11:01:51.123124 (+ 280us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52734: Request ticket server kudu/127.24.159.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":250,"threads_started":1}
W20260925 11:01:51.123493 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1724: Not authorized: Client connection negotiation failed: client connection to 127.24.159.194:37187: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.159.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 11:01:51.123535 2034 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52734: Request ticket server kudu/127.24.159.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 11:01:51.123623 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 1724
I20260925 11:01:51.136564 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1855
I20260925 11:01:51.147401 2035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:51.137426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.24.159.195:41863)
0925 11:01:51.137910 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:51.137918 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:51.137948 (+ 30us) server_negotiation.cc:407] Connection header received
0925 11:01:51.138020 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:51.138027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:51.138291 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:51.138472 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:51.139662 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:51.140620 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:51.141744 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:51.142353 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:51.142454 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:51.144601 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:51.144610 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:51.144622 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:51.144651 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:51.146938 (+ 2287us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.159.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0925 11:01:51.147171 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40948: Request ticket server kudu/127.24.159.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":216,"threads_started":1}
W20260925 11:01:51.147516 2035 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40948: Request ticket server kudu/127.24.159.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260925 11:01:51.147521 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1855: Not authorized: Client connection negotiation failed: client connection to 127.24.159.195:41863: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.159.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260925 11:01:51.147683 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 1855
I20260925 11:01:51.161128 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 1504
I20260925 11:01:51.162396 1566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:51.277433 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 1504
2026-09-25T11:01:51Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20697 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-25T11:01:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:51Z Disabled control of system clock
I20260925 11:01:51.344529 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:42087
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:42635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:42087
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260925 11:01:51.657056 2040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:51.657332 2040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:51.657446 2040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:51.664536 2040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:51.664640 2040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:51.664700 2040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:51.664786 2040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:51.664842 2040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:51.673893 2040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:42087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:42087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:51.675179 2040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:51.677107 2040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:51.689250 2045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:51.689205 2048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:51.689277 2046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:51.689754 2040 server_base.cc:1034] running on GCE node
I20260925 11:01:51.690698 2040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:51.692095 2040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:51.693285 2040 hybrid_clock.cc:648] HybridClock initialized: now 1790334111693242 us; error 45 us; skew 500 ppm
I20260925 11:01:51.693605 2040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:51.695742 2040 webserver.cc:460] Webserver started at http://127.24.159.254:34367/ using document root <none> and password file <none>
I20260925 11:01:51.696419 2040 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:51.696516 2040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:51.696878 2040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:51.699753 2040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "ebf0f07a29284ed7a5f699539d531c3a"
format_stamp: "Formatted at 2026-09-25 11:01:51 on dist-test-slave-trms"
server_key: "ce0403893db5d23fa06c0a7a9681c572"
server_key_iv: "5ad22ea835e9de87d4778cfd152cee90"
server_key_version: "encryptionkey@0"
I20260925 11:01:51.700524 2040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "ebf0f07a29284ed7a5f699539d531c3a"
format_stamp: "Formatted at 2026-09-25 11:01:51 on dist-test-slave-trms"
server_key: "ce0403893db5d23fa06c0a7a9681c572"
server_key_iv: "5ad22ea835e9de87d4778cfd152cee90"
server_key_version: "encryptionkey@0"
I20260925 11:01:51.705545 2040 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260925 11:01:51.708876 2054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:51.710332 2040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:01:51.710515 2040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "ebf0f07a29284ed7a5f699539d531c3a"
format_stamp: "Formatted at 2026-09-25 11:01:51 on dist-test-slave-trms"
server_key: "ce0403893db5d23fa06c0a7a9681c572"
server_key_iv: "5ad22ea835e9de87d4778cfd152cee90"
server_key_version: "encryptionkey@0"
I20260925 11:01:51.710722 2040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:51.727638 2040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:51.728433 2040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:51.728688 2040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:51.744943 2040 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:42087
I20260925 11:01:51.744962 2105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:42087 every 8 connection(s)
I20260925 11:01:51.746237 2040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:51.749775 2106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:51.753132 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 2040
I20260925 11:01:51.753301 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:51.753928 25215 external_mini_cluster.cc:1428] Setting key e42e29a3179ff8158a462050bcabef58
I20260925 11:01:51.760550 2106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a: Bootstrap starting.
I20260925 11:01:51.764047 2106 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:51.765266 2106 log.cc:824] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:51.768146 2109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:51.755931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60772 (local address 127.24.159.254:42087)
0925 11:01:51.756867 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:51.756878 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:51.756925 (+ 47us) server_negotiation.cc:407] Connection header received
0925 11:01:51.759109 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:51.759132 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:51.759595 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:51.759904 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:51.760559 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:51.761825 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:51.763665 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:51.764394 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:51.764534 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:51.765638 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:51.765646 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:51.765660 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:51.766427 (+ 767us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:51.766535 (+ 108us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:51.766562 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:51.766677 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:51.766879 (+ 202us) server_negotiation.cc:299] Negotiation successful
0925 11:01:51.767204 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":324,"threads_started":1}
I20260925 11:01:51.768442 2106 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a: No bootstrap required, opened a new log
I20260925 11:01:51.775662 2106 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf0f07a29284ed7a5f699539d531c3a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42087 } }
I20260925 11:01:51.776024 2106 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:51.776114 2106 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebf0f07a29284ed7a5f699539d531c3a, State: Initialized, Role: FOLLOWER
I20260925 11:01:51.776742 2106 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf0f07a29284ed7a5f699539d531c3a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42087 } }
I20260925 11:01:51.776964 2106 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:51.777122 2106 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:51.777272 2106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:51.779398 2106 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf0f07a29284ed7a5f699539d531c3a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42087 } }
I20260925 11:01:51.779873 2106 leader_election.cc:302] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebf0f07a29284ed7a5f699539d531c3a; no voters:
I20260925 11:01:51.780397 2106 leader_election.cc:288] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:51.780560 2111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:51.781143 2111 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [term 1 LEADER]: Becoming Leader. State: Replica: ebf0f07a29284ed7a5f699539d531c3a, State: Running, Role: LEADER
I20260925 11:01:51.781702 2111 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf0f07a29284ed7a5f699539d531c3a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42087 } }
I20260925 11:01:51.782495 2106 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:51.784502 2112 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebf0f07a29284ed7a5f699539d531c3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf0f07a29284ed7a5f699539d531c3a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42087 } } }
I20260925 11:01:51.784621 2113 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebf0f07a29284ed7a5f699539d531c3a. Latest consensus state: current_term: 1 leader_uuid: "ebf0f07a29284ed7a5f699539d531c3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebf0f07a29284ed7a5f699539d531c3a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 42087 } } }
I20260925 11:01:51.784879 2112 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:51.784875 2113 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:51.786305 2119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:51.791469 2119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:51.799947 2119 catalog_manager.cc:1347] Generated new cluster ID: 885076d589ed444e9c1d8094092f019c
I20260925 11:01:51.800071 2119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:51.816092 2119 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:51.817559 2119 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:51.834245 2119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebf0f07a29284ed7a5f699539d531c3a: Generated new TSK 0
I20260925 11:01:51.835364 2119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:01:51.841063 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42087
--builtin_ntp_servers=127.24.159.212:42635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260925 11:01:52.159139 2130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:52.159423 2130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:52.159540 2130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:52.166595 2130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:52.166705 2130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:52.166847 2130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:52.175791 2130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:52.177073 2130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:52.178824 2130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:52.191308 2135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:52.191262 2138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:52.191332 2136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:52.192149 2130 server_base.cc:1034] running on GCE node
I20260925 11:01:52.192718 2130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:52.193620 2130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:52.194850 2130 hybrid_clock.cc:648] HybridClock initialized: now 1790334112194805 us; error 31 us; skew 500 ppm
I20260925 11:01:52.195168 2130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:52.197304 2130 webserver.cc:460] Webserver started at http://127.24.159.193:34963/ using document root <none> and password file <none>
I20260925 11:01:52.197966 2130 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:52.198053 2130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:52.198375 2130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:52.201159 2130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "b4810cb0054047d9a96d2dd2a9c21714"
format_stamp: "Formatted at 2026-09-25 11:01:52 on dist-test-slave-trms"
server_key: "c4fc5f694a1c8cab83bd20d41b40983b"
server_key_iv: "c84f0db33f22bd46b7b02a141942132a"
server_key_version: "encryptionkey@0"
I20260925 11:01:52.201889 2130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "b4810cb0054047d9a96d2dd2a9c21714"
format_stamp: "Formatted at 2026-09-25 11:01:52 on dist-test-slave-trms"
server_key: "c4fc5f694a1c8cab83bd20d41b40983b"
server_key_iv: "c84f0db33f22bd46b7b02a141942132a"
server_key_version: "encryptionkey@0"
I20260925 11:01:52.206827 2130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 11:01:52.210026 2144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:52.211344 2130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:01:52.211475 2130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "b4810cb0054047d9a96d2dd2a9c21714"
format_stamp: "Formatted at 2026-09-25 11:01:52 on dist-test-slave-trms"
server_key: "c4fc5f694a1c8cab83bd20d41b40983b"
server_key_iv: "c84f0db33f22bd46b7b02a141942132a"
server_key_version: "encryptionkey@0"
I20260925 11:01:52.211647 2130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:52.239362 2130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:52.240190 2130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:52.240443 2130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:52.241343 2130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:52.242846 2130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:52.242929 2130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:52.243006 2130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:52.243060 2130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:52.271327 2130 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:44979
I20260925 11:01:52.271414 2256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:44979 every 8 connection(s)
I20260925 11:01:52.272568 2130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:52.282063 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 2130
I20260925 11:01:52.282310 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:01:52.282955 25215 external_mini_cluster.cc:1428] Setting key eed675436036a681a9970afe316ab211
I20260925 11:01:52.284232 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:52.274907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:35433 (local address 127.24.159.254:42087)
0925 11:01:52.275354 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:52.275361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:52.276532 (+ 1171us) server_negotiation.cc:407] Connection header received
0925 11:01:52.278165 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:52.278169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:52.278263 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:52.278355 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:52.279333 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:52.280033 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:52.281455 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:52.281989 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:52.282111 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:52.283399 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:52.283405 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:52.283409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:52.283431 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:52.283479 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:52.283496 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:52.283617 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:52.283834 (+ 217us) server_negotiation.cc:299] Negotiation successful
0925 11:01:52.283909 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":203,"threads_started":1}
I20260925 11:01:52.285194 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:52.275525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:42087 (local address 127.24.159.193:35433)
0925 11:01:52.276390 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:52.276429 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:52.277943 (+ 1514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:52.278547 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:52.278555 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:52.278649 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:52.279206 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:52.279222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:52.280145 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:52.280148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:52.281342 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:52.281353 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:52.282168 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:52.282175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:52.282445 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:52.283106 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:52.283109 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:52.283152 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:52.283226 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:52.283597 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:52.283607 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:52.283616 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 11:01:52.283805 (+ 189us) client_negotiation.cc:240] Negotiation successful
0925 11:01:52.284133 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":238,"threads_started":1}
I20260925 11:01:52.285753 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42087
--builtin_ntp_servers=127.24.159.212:42635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 11:01:52.286634 2257 heartbeater.cc:347] Connected to a master server at 127.24.159.254:42087
I20260925 11:01:52.286911 2257 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:52.287599 2257 heartbeater.cc:511] Master 127.24.159.254:42087 requested a full tablet report, sending...
I20260925 11:01:52.289482 2071 ts_manager.cc:194] Registered new tserver with Master: b4810cb0054047d9a96d2dd2a9c21714 (127.24.159.193:44979)
I20260925 11:01:52.290956 2071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.193:35433
W20260925 11:01:52.599192 2261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:52.599473 2261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:52.599586 2261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:52.606803 2261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:52.606902 2261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:52.607040 2261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:52.616310 2261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:52.617563 2261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:52.619299 2261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:52.631697 2266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:52.631728 2267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:52.631783 2269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:52.632601 2261 server_base.cc:1034] running on GCE node
I20260925 11:01:52.633177 2261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:52.634063 2261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:52.635260 2261 hybrid_clock.cc:648] HybridClock initialized: now 1790334112635207 us; error 59 us; skew 500 ppm
I20260925 11:01:52.635561 2261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:52.637714 2261 webserver.cc:460] Webserver started at http://127.24.159.194:33377/ using document root <none> and password file <none>
I20260925 11:01:52.638446 2261 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:52.638547 2261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:52.638849 2261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:52.641623 2261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "4a73d7d2c1274b9287deb6015a72f0e9"
format_stamp: "Formatted at 2026-09-25 11:01:52 on dist-test-slave-trms"
server_key: "d17ac2524ca0ca3b8f8a3d4689768565"
server_key_iv: "dd590da7664e1300bbe8b64ce6227d7f"
server_key_version: "encryptionkey@0"
I20260925 11:01:52.642436 2261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "4a73d7d2c1274b9287deb6015a72f0e9"
format_stamp: "Formatted at 2026-09-25 11:01:52 on dist-test-slave-trms"
server_key: "d17ac2524ca0ca3b8f8a3d4689768565"
server_key_iv: "dd590da7664e1300bbe8b64ce6227d7f"
server_key_version: "encryptionkey@0"
I20260925 11:01:52.647447 2261 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:01:52.651010 2275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:52.652341 2261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:01:52.652501 2261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "4a73d7d2c1274b9287deb6015a72f0e9"
format_stamp: "Formatted at 2026-09-25 11:01:52 on dist-test-slave-trms"
server_key: "d17ac2524ca0ca3b8f8a3d4689768565"
server_key_iv: "dd590da7664e1300bbe8b64ce6227d7f"
server_key_version: "encryptionkey@0"
I20260925 11:01:52.652701 2261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:52.698422 2261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:52.699313 2261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:52.699587 2261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:52.700551 2261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:52.702014 2261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:52.702107 2261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:52.702229 2261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:52.702306 2261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:52.732291 2261 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:37835
I20260925 11:01:52.732335 2387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:37835 every 8 connection(s)
I20260925 11:01:52.733553 2261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:52.737010 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 2261
I20260925 11:01:52.737167 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:01:52.737620 25215 external_mini_cluster.cc:1428] Setting key fb50e878668ae011a5a0176ca35caf4f
I20260925 11:01:52.740257 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42087
--builtin_ntp_servers=127.24.159.212:42635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260925 11:01:52.746893 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:52.735977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:43497 (local address 127.24.159.254:42087)
0925 11:01:52.736190 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:52.736196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:52.737747 (+ 1551us) server_negotiation.cc:407] Connection header received
0925 11:01:52.739701 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:52.739707 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:52.739787 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:52.739894 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:52.741099 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:52.742084 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:52.743738 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:52.744300 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:52.744442 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:52.746041 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:52.746046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:52.746050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:52.746072 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:52.746113 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:52.746130 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:52.746245 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:52.746480 (+ 235us) server_negotiation.cc:299] Negotiation successful
0925 11:01:52.746584 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 11:01:52.748189 2390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:52.736550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:42087 (local address 127.24.159.194:43497)
0925 11:01:52.737582 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:52.737628 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:52.739431 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:52.740141 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:52.740150 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:52.740253 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:52.740923 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:52.740942 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:52.742295 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:52.742300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:52.743587 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:52.743600 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:52.744537 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:52.744545 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:52.744840 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:52.745648 (+ 808us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:52.745651 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:52.745714 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:52.745818 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:52.746257 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:52.746268 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:52.746277 (+ 9us) client_negotiation.cc:769] Sending connection context
0925 11:01:52.746473 (+ 196us) client_negotiation.cc:240] Negotiation successful
0925 11:01:52.746907 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":300,"threads_started":1}
I20260925 11:01:52.749797 2388 heartbeater.cc:347] Connected to a master server at 127.24.159.254:42087
I20260925 11:01:52.750115 2388 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:52.751065 2388 heartbeater.cc:511] Master 127.24.159.254:42087 requested a full tablet report, sending...
I20260925 11:01:52.752612 2071 ts_manager.cc:194] Registered new tserver with Master: 4a73d7d2c1274b9287deb6015a72f0e9 (127.24.159.194:37835)
I20260925 11:01:52.753688 2071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.194:43497
W20260925 11:01:53.060496 2391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:53.060776 2391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:53.060900 2391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:53.067981 2391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:53.068086 2391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:53.068228 2391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:53.077067 2391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:42087
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:53.078323 2391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:53.080013 2391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:53.092239 2397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:53.092383 2396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:53.092231 2399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:53.092834 2391 server_base.cc:1034] running on GCE node
I20260925 11:01:53.093453 2391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:53.094372 2391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:53.095568 2391 hybrid_clock.cc:648] HybridClock initialized: now 1790334113095515 us; error 50 us; skew 500 ppm
I20260925 11:01:53.095857 2391 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:53.098006 2391 webserver.cc:460] Webserver started at http://127.24.159.195:45195/ using document root <none> and password file <none>
I20260925 11:01:53.098711 2391 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:53.098805 2391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:53.099097 2391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:53.101791 2391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "ccbdaba9cf1342a19d58e7727cb2799a"
format_stamp: "Formatted at 2026-09-25 11:01:53 on dist-test-slave-trms"
server_key: "7a18caa542efbfacf857175362e06993"
server_key_iv: "77c14348bedc9daa0291090329b8cfdd"
server_key_version: "encryptionkey@0"
I20260925 11:01:53.102551 2391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "ccbdaba9cf1342a19d58e7727cb2799a"
format_stamp: "Formatted at 2026-09-25 11:01:53 on dist-test-slave-trms"
server_key: "7a18caa542efbfacf857175362e06993"
server_key_iv: "77c14348bedc9daa0291090329b8cfdd"
server_key_version: "encryptionkey@0"
I20260925 11:01:53.107260 2391 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260925 11:01:53.110513 2405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:53.111763 2391 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:01:53.111909 2391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "ccbdaba9cf1342a19d58e7727cb2799a"
format_stamp: "Formatted at 2026-09-25 11:01:53 on dist-test-slave-trms"
server_key: "7a18caa542efbfacf857175362e06993"
server_key_iv: "77c14348bedc9daa0291090329b8cfdd"
server_key_version: "encryptionkey@0"
I20260925 11:01:53.112097 2391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:53.123744 2391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:53.124567 2391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:53.124823 2391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:53.125725 2391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:53.127183 2391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:53.127274 2391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:53.127383 2391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:53.127450 2391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:53.154683 2391 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:35789
I20260925 11:01:53.154729 2517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:35789 every 8 connection(s)
I20260925 11:01:53.155972 2391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:53.164072 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 2391
I20260925 11:01:53.164217 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:01:53.164635 25215 external_mini_cluster.cc:1428] Setting key 5032e08f68c59586d27d3d7948ca43b9
I20260925 11:01:53.167910 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:53.158371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:35483 (local address 127.24.159.254:42087)
0925 11:01:53.158568 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:53.158574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:53.159935 (+ 1361us) server_negotiation.cc:407] Connection header received
0925 11:01:53.161582 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:53.161586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:53.161656 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:53.161730 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:53.162770 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.163482 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.165051 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.165541 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.165649 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:53.167189 (+ 1540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:53.167193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:53.167197 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:53.167213 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:53.167244 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:53.167255 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:53.167320 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:53.167554 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 11:01:53.167648 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:01:53.168889 2520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:53.158937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:42087 (local address 127.24.159.195:35483)
0925 11:01:53.159781 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:53.159816 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:53.161339 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:53.161928 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:53.161935 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:53.162030 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:53.162640 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:53.162655 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.163592 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.163595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:53.164915 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:53.164926 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.165782 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.165790 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:53.166054 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:53.166851 (+ 797us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:53.166854 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:53.166920 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:53.167022 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:53.167345 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:53.167364 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:53.167371 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 11:01:53.167523 (+ 152us) client_negotiation.cc:240] Negotiation successful
0925 11:01:53.167882 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":241,"threads_started":1}
I20260925 11:01:53.170150 2518 heartbeater.cc:347] Connected to a master server at 127.24.159.254:42087
I20260925 11:01:53.170436 2518 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:53.171151 2518 heartbeater.cc:511] Master 127.24.159.254:42087 requested a full tablet report, sending...
I20260925 11:01:53.172415 2071 ts_manager.cc:194] Registered new tserver with Master: ccbdaba9cf1342a19d58e7727cb2799a (127.24.159.195:35789)
I20260925 11:01:53.173152 2071 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.195:35483
I20260925 11:01:53.179776 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 11:01:53.185596 2259 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:53.184032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60778 (local address 127.24.159.254:42087)
0925 11:01:53.184189 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:53.184195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:53.184610 (+ 415us) server_negotiation.cc:407] Connection header received
0925 11:01:53.184766 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:53.184770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:53.184841 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:53.184924 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:53.185348 (+ 424us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60778: BlockingRecv error: recv got EOF from 127.0.0.1:60778 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 11:01:53.186172 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 2130
I20260925 11:01:53.193329 2527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:53.187022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43882 (local address 127.24.159.193:44979)
0925 11:01:53.187515 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:53.187523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:53.187548 (+ 25us) server_negotiation.cc:407] Connection header received
0925 11:01:53.187601 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:53.187607 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:53.187818 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:53.187944 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:53.188256 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.189280 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.190492 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.191038 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.191132 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:53.192057 (+ 925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:53.192065 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:53.192078 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:53.192650 (+ 572us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:53.192694 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:53.192712 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:53.192810 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:53.192884 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 11:01:53.192998 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":246,"threads_started":1}
I20260925 11:01:53.194504 2252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:53.274909 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 2130
I20260925 11:01:53.288475 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 2261
I20260925 11:01:53.301528 2529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:53.289397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56544 (local address 127.24.159.194:37835)
0925 11:01:53.289918 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:53.289925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:53.289951 (+ 26us) server_negotiation.cc:407] Connection header received
0925 11:01:53.290015 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:53.290021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:53.290296 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:53.290450 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:53.290823 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.291812 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.293019 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.293561 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.293670 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:53.300115 (+ 6445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:53.300124 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:53.300138 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:53.300737 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:53.300796 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:53.300818 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:53.300928 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:53.301068 (+ 140us) server_negotiation.cc:299] Negotiation successful
0925 11:01:53.301210 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":278,"threads_started":1}
I20260925 11:01:53.302831 2383 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:53.381850 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 2261
I20260925 11:01:53.394820 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 2391
I20260925 11:01:53.402473 2531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:53.395733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.24.159.195:35789)
0925 11:01:53.396311 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:53.396319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:53.396349 (+ 30us) server_negotiation.cc:407] Connection header received
0925 11:01:53.396404 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:53.396411 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:53.396646 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:53.396771 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:53.397184 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.398265 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.399510 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:53.400054 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:53.400145 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:53.401056 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:53.401064 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:53.401079 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:53.401671 (+ 592us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:53.401718 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:53.401739 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:53.401831 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:53.401991 (+ 160us) server_negotiation.cc:299] Negotiation successful
0925 11:01:53.402116 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":302,"threads_started":1}
I20260925 11:01:53.403697 2513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:53.484056 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 2391
I20260925 11:01:53.497141 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 2040
I20260925 11:01:53.498279 2101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:53.597470 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 2040
2026-09-25T11:01:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2315 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-25T11:01:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:53Z Disabled control of system clock
I20260925 11:01:53.658977 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:39979
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:43361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:39979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 11:01:53.969130 2537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:53.969416 2537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:53.969544 2537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:53.976586 2537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:53.976691 2537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:53.976748 2537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:53.976822 2537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:53.976876 2537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:53.985888 2537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:43361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:39979
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:39979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:53.987214 2537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:53.988956 2537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:53.999763 2542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:53.999882 2543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:53.999817 2545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:54.000909 2537 server_base.cc:1034] running on GCE node
I20260925 11:01:54.001644 2537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:54.003072 2537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:54.004276 2537 hybrid_clock.cc:648] HybridClock initialized: now 1790334114004223 us; error 44 us; skew 500 ppm
I20260925 11:01:54.004594 2537 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:54.006912 2537 webserver.cc:460] Webserver started at http://127.24.159.254:42801/ using document root <none> and password file <none>
I20260925 11:01:54.007561 2537 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:54.007656 2537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:54.007957 2537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:54.010828 2537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "2924484618144a07b1434039096693f1"
format_stamp: "Formatted at 2026-09-25 11:01:54 on dist-test-slave-trms"
server_key: "eb0851a7c985d70896231ecaba35f13d"
server_key_iv: "f67ca6b2a1177ce6a0a37b2d36989927"
server_key_version: "encryptionkey@0"
I20260925 11:01:54.011579 2537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "2924484618144a07b1434039096693f1"
format_stamp: "Formatted at 2026-09-25 11:01:54 on dist-test-slave-trms"
server_key: "eb0851a7c985d70896231ecaba35f13d"
server_key_iv: "f67ca6b2a1177ce6a0a37b2d36989927"
server_key_version: "encryptionkey@0"
I20260925 11:01:54.016484 2537 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:01:54.019757 2551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:54.021019 2537 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:01:54.021171 2537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "2924484618144a07b1434039096693f1"
format_stamp: "Formatted at 2026-09-25 11:01:54 on dist-test-slave-trms"
server_key: "eb0851a7c985d70896231ecaba35f13d"
server_key_iv: "f67ca6b2a1177ce6a0a37b2d36989927"
server_key_version: "encryptionkey@0"
I20260925 11:01:54.021366 2537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:54.027015 2537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:54.027839 2537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:54.028105 2537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:54.045323 2537 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:39979
I20260925 11:01:54.045352 2602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:39979 every 8 connection(s)
I20260925 11:01:54.046629 2537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:54.050290 2603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:54.056566 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 2537
I20260925 11:01:54.056775 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:54.057368 25215 external_mini_cluster.cc:1428] Setting key c1227b8de3affd22bc0934e0901fdb17
I20260925 11:01:54.060334 2603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1: Bootstrap starting.
I20260925 11:01:54.063586 2603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:54.064779 2603 log.cc:824] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:54.065341 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:54.059250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.24.159.254:39979)
0925 11:01:54.060195 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:54.060206 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:54.060253 (+ 47us) server_negotiation.cc:407] Connection header received
0925 11:01:54.061866 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:54.061885 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:54.062340 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:54.062646 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:54.063726 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:54.063734 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:54.063749 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:54.063767 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:54.063830 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:54.063846 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:54.064001 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:54.064166 (+ 165us) server_negotiation.cc:299] Negotiation successful
0925 11:01:54.064433 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":310,"threads_started":1}
I20260925 11:01:54.067790 2603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1: No bootstrap required, opened a new log
I20260925 11:01:54.074765 2603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2924484618144a07b1434039096693f1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39979 } }
I20260925 11:01:54.075093 2603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:54.075183 2603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2924484618144a07b1434039096693f1, State: Initialized, Role: FOLLOWER
I20260925 11:01:54.075706 2603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2924484618144a07b1434039096693f1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39979 } }
I20260925 11:01:54.075904 2603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:54.076030 2603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:54.076174 2603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:54.078410 2603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2924484618144a07b1434039096693f1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39979 } }
I20260925 11:01:54.078887 2603 leader_election.cc:302] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2924484618144a07b1434039096693f1; no voters:
I20260925 11:01:54.079452 2603 leader_election.cc:288] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:54.079572 2608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:54.080255 2608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [term 1 LEADER]: Becoming Leader. State: Replica: 2924484618144a07b1434039096693f1, State: Running, Role: LEADER
I20260925 11:01:54.080751 2608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2924484618144a07b1434039096693f1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39979 } }
I20260925 11:01:54.081321 2603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:54.083418 2610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2924484618144a07b1434039096693f1. Latest consensus state: current_term: 1 leader_uuid: "2924484618144a07b1434039096693f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2924484618144a07b1434039096693f1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39979 } } }
I20260925 11:01:54.083782 2610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:54.084080 2609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2924484618144a07b1434039096693f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2924484618144a07b1434039096693f1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39979 } } }
I20260925 11:01:54.084455 2609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:54.086073 2617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:54.090330 2617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:54.098099 2617 catalog_manager.cc:1347] Generated new cluster ID: 290f069dc3e04e1a9f456dabfe952ae2
I20260925 11:01:54.098227 2617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:54.117884 2617 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:54.118348 2617 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:54.128489 2617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2924484618144a07b1434039096693f1: Generated new TSK 0
I20260925 11:01:54.129294 2617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:01:54.137697 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:39979
--builtin_ntp_servers=127.24.159.212:43361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260925 11:01:54.456393 2627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:54.456692 2627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:54.456806 2627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:54.464089 2627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:54.464203 2627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:54.464334 2627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:54.473312 2627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:43361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:39979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:54.474618 2627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:54.476367 2627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:54.489018 2635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:54.489045 2632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:54.489018 2633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:54.489832 2627 server_base.cc:1034] running on GCE node
I20260925 11:01:54.490440 2627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:54.491353 2627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:54.492553 2627 hybrid_clock.cc:648] HybridClock initialized: now 1790334114492498 us; error 42 us; skew 500 ppm
I20260925 11:01:54.492879 2627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:54.495229 2627 webserver.cc:460] Webserver started at http://127.24.159.193:36629/ using document root <none> and password file <none>
I20260925 11:01:54.495910 2627 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:54.496012 2627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:54.496320 2627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:54.499248 2627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "0d409aa5e37a4007b90e6f615c3f125b"
format_stamp: "Formatted at 2026-09-25 11:01:54 on dist-test-slave-trms"
server_key: "d12916655e257ede0a79cd8771fc8753"
server_key_iv: "8f6f117eecb360949a3a34fd1779cba3"
server_key_version: "encryptionkey@0"
I20260925 11:01:54.500031 2627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "0d409aa5e37a4007b90e6f615c3f125b"
format_stamp: "Formatted at 2026-09-25 11:01:54 on dist-test-slave-trms"
server_key: "d12916655e257ede0a79cd8771fc8753"
server_key_iv: "8f6f117eecb360949a3a34fd1779cba3"
server_key_version: "encryptionkey@0"
I20260925 11:01:54.505191 2627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 11:01:54.508594 2641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:54.509984 2627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 11:01:54.510119 2627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "0d409aa5e37a4007b90e6f615c3f125b"
format_stamp: "Formatted at 2026-09-25 11:01:54 on dist-test-slave-trms"
server_key: "d12916655e257ede0a79cd8771fc8753"
server_key_iv: "8f6f117eecb360949a3a34fd1779cba3"
server_key_version: "encryptionkey@0"
I20260925 11:01:54.510334 2627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:54.515640 2627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:54.516386 2627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:54.516599 2627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:54.517423 2627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:54.518851 2627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:54.518944 2627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:54.519052 2627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:54.519129 2627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:54.548267 2627 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:46767
I20260925 11:01:54.548324 2753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:46767 every 8 connection(s)
I20260925 11:01:54.549597 2627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:54.550937 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 2627
I20260925 11:01:54.551126 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:01:54.551805 25215 external_mini_cluster.cc:1428] Setting key fb033c4f740f54f42053e7ad5bd6ad79
I20260925 11:01:54.554487 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:39979
--builtin_ntp_servers=127.24.159.212:43361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 11:01:54.557682 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:54.552185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:60881 (local address 127.24.159.254:39979)
0925 11:01:54.552350 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:54.552355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:54.554025 (+ 1670us) server_negotiation.cc:407] Connection header received
0925 11:01:54.555700 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:54.555705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:54.555762 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:54.555861 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:54.557058 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:54.557063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:54.557066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:54.557084 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:54.557121 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:54.557134 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:54.557245 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:54.557384 (+ 139us) server_negotiation.cc:299] Negotiation successful
0925 11:01:54.557442 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:01:54.558632 2756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:54.552819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:39979 (local address 127.24.159.193:60881)
0925 11:01:54.553855 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:54.553900 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:54.555439 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:54.556057 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:54.556065 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:54.556162 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:54.556760 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:54.556762 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:54.556800 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:54.556884 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:54.557216 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:54.557222 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:54.557227 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 11:01:54.557423 (+ 196us) client_negotiation.cc:240] Negotiation successful
0925 11:01:54.557621 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":299,"threads_started":1}
I20260925 11:01:54.560048 2754 heartbeater.cc:347] Connected to a master server at 127.24.159.254:39979
I20260925 11:01:54.560355 2754 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:54.561010 2754 heartbeater.cc:511] Master 127.24.159.254:39979 requested a full tablet report, sending...
I20260925 11:01:54.562985 2568 ts_manager.cc:194] Registered new tserver with Master: 0d409aa5e37a4007b90e6f615c3f125b (127.24.159.193:46767)
W20260925 11:01:54.871650 2757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:54.871927 2757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:54.872038 2757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:54.879114 2757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:54.879208 2757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:54.879352 2757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:54.888537 2757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:43361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:39979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:54.889755 2757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:54.891487 2757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:54.903250 2762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:54.903308 2763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:54.903601 2765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:54.904654 2757 server_base.cc:1034] running on GCE node
I20260925 11:01:54.905190 2757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:54.906080 2757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:54.907292 2757 hybrid_clock.cc:648] HybridClock initialized: now 1790334114907248 us; error 51 us; skew 500 ppm
I20260925 11:01:54.907598 2757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:54.909652 2757 webserver.cc:460] Webserver started at http://127.24.159.194:38079/ using document root <none> and password file <none>
I20260925 11:01:54.910351 2757 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:54.910447 2757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:54.910734 2757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:54.914779 2757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "23bffde8686149c1a4f6bc19a47f99a7"
format_stamp: "Formatted at 2026-09-25 11:01:54 on dist-test-slave-trms"
server_key: "45cf04cacdd112a29de6567b4c0b3b9a"
server_key_iv: "708637ffd37a054d6db67dfb34866704"
server_key_version: "encryptionkey@0"
I20260925 11:01:54.915514 2757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "23bffde8686149c1a4f6bc19a47f99a7"
format_stamp: "Formatted at 2026-09-25 11:01:54 on dist-test-slave-trms"
server_key: "45cf04cacdd112a29de6567b4c0b3b9a"
server_key_iv: "708637ffd37a054d6db67dfb34866704"
server_key_version: "encryptionkey@0"
I20260925 11:01:54.920341 2757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 11:01:54.923687 2771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:54.925132 2757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:01:54.925284 2757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "23bffde8686149c1a4f6bc19a47f99a7"
format_stamp: "Formatted at 2026-09-25 11:01:54 on dist-test-slave-trms"
server_key: "45cf04cacdd112a29de6567b4c0b3b9a"
server_key_iv: "708637ffd37a054d6db67dfb34866704"
server_key_version: "encryptionkey@0"
I20260925 11:01:54.925510 2757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:54.931106 2757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:54.931903 2757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:54.932142 2757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:54.933058 2757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:54.934432 2757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:54.934525 2757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:54.934628 2757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:54.934696 2757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:54.963554 2757 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:44721
I20260925 11:01:54.963606 2883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:44721 every 8 connection(s)
I20260925 11:01:54.964804 2757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:54.965880 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 2757
I20260925 11:01:54.966065 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:01:54.966617 25215 external_mini_cluster.cc:1428] Setting key 6fe52ee0e7fb3888b7cc7c51662111b0
I20260925 11:01:54.969465 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:39979
--builtin_ntp_servers=127.24.159.212:43361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260925 11:01:54.972827 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:54.967405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:46285 (local address 127.24.159.254:39979)
0925 11:01:54.967563 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:54.967568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:54.969114 (+ 1546us) server_negotiation.cc:407] Connection header received
0925 11:01:54.970789 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:54.970794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:54.970859 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:54.970944 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:54.972155 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:54.972161 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:54.972166 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:54.972182 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:54.972213 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:54.972224 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:54.972332 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:54.972495 (+ 163us) server_negotiation.cc:299] Negotiation successful
0925 11:01:54.972559 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 11:01:54.973706 2886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:54.968004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:39979 (local address 127.24.159.194:46285)
0925 11:01:54.968975 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:54.969011 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:54.970539 (+ 1528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:54.971165 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:54.971173 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:54.971265 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:54.971860 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:54.971862 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:54.971903 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:54.971996 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:54.972304 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:54.972314 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:54.972322 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 11:01:54.972564 (+ 242us) client_negotiation.cc:240] Negotiation successful
0925 11:01:54.972806 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":329,"threads_started":1}
I20260925 11:01:54.974984 2884 heartbeater.cc:347] Connected to a master server at 127.24.159.254:39979
I20260925 11:01:54.975241 2884 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:54.975895 2884 heartbeater.cc:511] Master 127.24.159.254:39979 requested a full tablet report, sending...
I20260925 11:01:54.977233 2568 ts_manager.cc:194] Registered new tserver with Master: 23bffde8686149c1a4f6bc19a47f99a7 (127.24.159.194:44721)
W20260925 11:01:55.284554 2887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:55.284920 2887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:55.285056 2887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:55.292292 2887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:55.292384 2887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:55.292510 2887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:55.301452 2887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:43361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:39979
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:55.302709 2887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:55.304394 2887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:55.316996 2892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:55.317054 2893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:55.317103 2895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:55.317691 2887 server_base.cc:1034] running on GCE node
I20260925 11:01:55.318296 2887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:55.319222 2887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:55.320443 2887 hybrid_clock.cc:648] HybridClock initialized: now 1790334115320375 us; error 56 us; skew 500 ppm
I20260925 11:01:55.320756 2887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:01:55.322896 2887 webserver.cc:460] Webserver started at http://127.24.159.195:34047/ using document root <none> and password file <none>
I20260925 11:01:55.323568 2887 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:55.323663 2887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:55.323992 2887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:55.326923 2887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "4fed7717d946444f93fdc301948f6b75"
format_stamp: "Formatted at 2026-09-25 11:01:55 on dist-test-slave-trms"
server_key: "5f03d11eab914cfa318bb38334e4d43b"
server_key_iv: "36907292b98e6d362e0373411368dc34"
server_key_version: "encryptionkey@0"
I20260925 11:01:55.327682 2887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "4fed7717d946444f93fdc301948f6b75"
format_stamp: "Formatted at 2026-09-25 11:01:55 on dist-test-slave-trms"
server_key: "5f03d11eab914cfa318bb38334e4d43b"
server_key_iv: "36907292b98e6d362e0373411368dc34"
server_key_version: "encryptionkey@0"
I20260925 11:01:55.332937 2887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:01:55.337512 2901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:55.339383 2887 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260925 11:01:55.339598 2887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "4fed7717d946444f93fdc301948f6b75"
format_stamp: "Formatted at 2026-09-25 11:01:55 on dist-test-slave-trms"
server_key: "5f03d11eab914cfa318bb38334e4d43b"
server_key_iv: "36907292b98e6d362e0373411368dc34"
server_key_version: "encryptionkey@0"
I20260925 11:01:55.339891 2887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:55.347052 2887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:55.347863 2887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:55.348114 2887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:55.349153 2887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:55.351074 2887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:55.351181 2887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:55.351291 2887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:55.351374 2887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:55.383261 2887 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:38129
I20260925 11:01:55.383307 3013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:38129 every 8 connection(s)
I20260925 11:01:55.384559 2887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:55.389979 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 2887
I20260925 11:01:55.390172 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:01:55.390753 25215 external_mini_cluster.cc:1428] Setting key 7529fb3481bb66d01ba199a91ecefe11
I20260925 11:01:55.393241 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:55.387136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:40197 (local address 127.24.159.254:39979)
0925 11:01:55.387316 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:55.387320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:55.388752 (+ 1432us) server_negotiation.cc:407] Connection header received
0925 11:01:55.390783 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:55.390787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:55.390858 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:55.390968 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:55.392503 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:55.392509 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:55.392513 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:55.392535 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:55.392577 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:55.392593 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:55.392686 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:55.392906 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 11:01:55.392979 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 11:01:55.394239 3016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:55.387679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:39979 (local address 127.24.159.195:40197)
0925 11:01:55.388583 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:55.388625 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:55.390465 (+ 1840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:55.391244 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:55.391256 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:55.391372 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:55.392132 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:01:55.392136 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:55.392191 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:01:55.392310 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:55.392685 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:55.392695 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:55.392703 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 11:01:55.392957 (+ 254us) client_negotiation.cc:240] Negotiation successful
0925 11:01:55.393173 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":260,"threads_started":1}
I20260925 11:01:55.395402 3014 heartbeater.cc:347] Connected to a master server at 127.24.159.254:39979
I20260925 11:01:55.395685 3014 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:55.396234 3014 heartbeater.cc:511] Master 127.24.159.254:39979 requested a full tablet report, sending...
I20260925 11:01:55.397509 2568 ts_manager.cc:194] Registered new tserver with Master: 4fed7717d946444f93fdc301948f6b75 (127.24.159.195:38129)
I20260925 11:01:55.406085 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260925 11:01:55.411931 2606 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:01:55.410554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.24.159.254:39979)
0925 11:01:55.410714 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:55.410719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:55.411162 (+ 443us) server_negotiation.cc:407] Connection header received
0925 11:01:55.411296 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:55.411299 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:55.411349 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:55.411442 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:55.411676 (+ 234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56240: BlockingRecv error: recv got EOF from 127.0.0.1:56240 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 11:01:55.412706 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 2627
I20260925 11:01:55.416224 3023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:55.413579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.24.159.193:46767)
0925 11:01:55.414076 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:55.414083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:55.414107 (+ 24us) server_negotiation.cc:407] Connection header received
0925 11:01:55.414159 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:55.414165 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:55.414449 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:55.414589 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:55.415536 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:55.415542 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:55.415555 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:55.415572 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:55.415610 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:55.415622 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:55.415737 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:55.415860 (+ 123us) server_negotiation.cc:299] Negotiation successful
0925 11:01:55.415961 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":239,"threads_started":1}
I20260925 11:01:55.417364 2749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:55.496481 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 2627
I20260925 11:01:55.509768 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 2757
I20260925 11:01:55.513048 3025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:55.510717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.24.159.194:44721)
0925 11:01:55.511204 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:55.511211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:55.511238 (+ 27us) server_negotiation.cc:407] Connection header received
0925 11:01:55.511299 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:55.511306 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:55.511544 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:55.511663 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:55.512501 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:55.512507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:55.512518 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:55.512535 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:55.512575 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:55.512589 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:55.512690 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:55.512739 (+ 49us) server_negotiation.cc:299] Negotiation successful
0925 11:01:55.512817 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":242,"threads_started":1}
I20260925 11:01:55.514243 2879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:55.593560 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 2757
I20260925 11:01:55.606297 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 2887
I20260925 11:01:55.609827 3027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:55.607240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47660 (local address 127.24.159.195:38129)
0925 11:01:55.607885 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:55.607893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:55.607934 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:01:55.607996 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:55.608002 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:55.608213 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:55.608332 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:55.609234 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:55.609240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:55.609252 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:01:55.609267 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:55.609305 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:01:55.609319 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:55.609443 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:55.609494 (+ 51us) server_negotiation.cc:299] Negotiation successful
0925 11:01:55.609574 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":350,"threads_started":1}
I20260925 11:01:55.611011 3009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:55.690172 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 2887
I20260925 11:01:55.704397 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 2537
I20260925 11:01:55.705546 2598 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:55.804571 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 2537
2026-09-25T11:01:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2206 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:01:55 dist-test-slave-trms krb5kdc[3032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:01:55 dist-test-slave-trms krb5kdc[3032](info): set up 2 sockets
Sep 25 11:01:55 dist-test-slave-trms krb5kdc[3032](info): commencing operation
krb5kdc: starting...
W20260925 11:01:56.815073 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.002s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:01:56 dist-test-slave-trms krb5kdc[3032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:01:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:01:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:01:57.029376 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37855
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:37855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:57.343410 3045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:57.343688 3045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:57.343808 3045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:57.350854 3045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:01:57.350958 3045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:57.351029 3045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:57.351099 3045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:01:57.351157 3045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:01:57.360664 3045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:37855
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:57.361967 3045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:57.363258 3045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:57.374209 3053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:57.374256 3051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:57.374256 3050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:57.374953 3045 server_base.cc:1034] running on GCE node
I20260925 11:01:57.375695 3045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:57.377018 3045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:57.378237 3045 hybrid_clock.cc:648] HybridClock initialized: now 1790334117378167 us; error 38 us; skew 500 ppm
I20260925 11:01:57.378561 3045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:57 dist-test-slave-trms krb5kdc[3032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334117, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:57.381582 3045 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:01:57.382970 3045 webserver.cc:460] Webserver started at http://127.24.159.254:35527/ using document root <none> and password file <none>
I20260925 11:01:57.383608 3045 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:57.383713 3045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:57.384002 3045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:57.386891 3045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "ffc28f6f85014dbebed9e5195a4550b7"
format_stamp: "Formatted at 2026-09-25 11:01:57 on dist-test-slave-trms"
server_key: "1f8ca9202864f773c8d4418f8833b480"
server_key_iv: "f0c070a0a732708d44036c248d37c218"
server_key_version: "encryptionkey@0"
I20260925 11:01:57.387589 3045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "ffc28f6f85014dbebed9e5195a4550b7"
format_stamp: "Formatted at 2026-09-25 11:01:57 on dist-test-slave-trms"
server_key: "1f8ca9202864f773c8d4418f8833b480"
server_key_iv: "f0c070a0a732708d44036c248d37c218"
server_key_version: "encryptionkey@0"
I20260925 11:01:57.392473 3045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260925 11:01:57.395689 3060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:57.396883 3045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260925 11:01:57.397032 3045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "ffc28f6f85014dbebed9e5195a4550b7"
format_stamp: "Formatted at 2026-09-25 11:01:57 on dist-test-slave-trms"
server_key: "1f8ca9202864f773c8d4418f8833b480"
server_key_iv: "f0c070a0a732708d44036c248d37c218"
server_key_version: "encryptionkey@0"
I20260925 11:01:57.397193 3045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:57.425657 3045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:57.429769 3045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:57.430027 3045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:57.446203 3045 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:37855
I20260925 11:01:57.446251 3111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:37855 every 8 connection(s)
I20260925 11:01:57.447497 3045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:01:57.449009 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 3045
I20260925 11:01:57.449227 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:01:57.449995 25215 external_mini_cluster.cc:1428] Setting key 35a6830a024edd59e2fe6ba5a2199eaa
I20260925 11:01:57.451975 3112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 11:01:57 dist-test-slave-trms krb5kdc[3032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:57.464740 3112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7: Bootstrap starting.
I20260925 11:01:57.467887 3112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:57.469113 3112 log.cc:824] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:57.471607 3115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:57.451956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.24.159.254:37855)
0925 11:01:57.452685 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:57.452697 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:57.452736 (+ 39us) server_negotiation.cc:407] Connection header received
0925 11:01:57.454297 (+ 1561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:57.454319 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:57.454725 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:57.455042 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:57.456768 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:57.458033 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:57.459673 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:57.460299 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:57.460436 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:57.464799 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:57.464807 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:57.464822 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:57.464859 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:57.467621 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:57.468438 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:57.468444 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:57.468450 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:57.468606 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:57.469020 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:57.469023 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:57.469025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:57.469550 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:57.469855 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:57.470280 (+ 425us) server_negotiation.cc:299] Negotiation successful
0925 11:01:57.470670 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":310,"threads_started":1}
I20260925 11:01:57.471951 3112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7: No bootstrap required, opened a new log
I20260925 11:01:57.479913 3112 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc28f6f85014dbebed9e5195a4550b7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37855 } }
I20260925 11:01:57.480247 3112 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:57.480332 3112 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc28f6f85014dbebed9e5195a4550b7, State: Initialized, Role: FOLLOWER
I20260925 11:01:57.480919 3112 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc28f6f85014dbebed9e5195a4550b7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37855 } }
I20260925 11:01:57.481190 3112 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:01:57.481328 3112 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:01:57.481523 3112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:57.483634 3112 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc28f6f85014dbebed9e5195a4550b7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37855 } }
I20260925 11:01:57.484104 3112 leader_election.cc:302] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffc28f6f85014dbebed9e5195a4550b7; no voters:
I20260925 11:01:57.484668 3112 leader_election.cc:288] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:01:57.484817 3117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:57.485419 3117 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [term 1 LEADER]: Becoming Leader. State: Replica: ffc28f6f85014dbebed9e5195a4550b7, State: Running, Role: LEADER
I20260925 11:01:57.485888 3117 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc28f6f85014dbebed9e5195a4550b7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37855 } }
I20260925 11:01:57.486634 3112 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:01:57.488696 3119 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffc28f6f85014dbebed9e5195a4550b7. Latest consensus state: current_term: 1 leader_uuid: "ffc28f6f85014dbebed9e5195a4550b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc28f6f85014dbebed9e5195a4550b7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37855 } } }
I20260925 11:01:57.489102 3119 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:57.489025 3118 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffc28f6f85014dbebed9e5195a4550b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc28f6f85014dbebed9e5195a4550b7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37855 } } }
I20260925 11:01:57.489311 3118 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7 [sys.catalog]: This master's current role is: LEADER
I20260925 11:01:57.490680 3126 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:01:57.495983 3126 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:01:57.504349 3126 catalog_manager.cc:1347] Generated new cluster ID: f8d19464dd9541c993feaec27d245656
I20260925 11:01:57.504458 3126 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:01:57.532706 3126 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:01:57.534118 3126 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:01:57.544425 3126 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7: Generated new TSK 0
I20260925 11:01:57.545233 3126 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:01:57.601262 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37855
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:57.916921 3138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:57.917207 3138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:57.917330 3138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:57.924436 3138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:57.924554 3138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:57.924703 3138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:01:57.934228 3138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:57.935580 3138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:57.936792 3138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:57.948980 3144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:57.948992 3143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:57.948971 3146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:57.949940 3138 server_base.cc:1034] running on GCE node
I20260925 11:01:57.950541 3138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:57.951421 3138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:57.952611 3138 hybrid_clock.cc:648] HybridClock initialized: now 1790334117952559 us; error 46 us; skew 500 ppm
I20260925 11:01:57.952914 3138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:57 dist-test-slave-trms krb5kdc[3032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334117, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:57.956003 3138 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:01:57.957442 3138 webserver.cc:460] Webserver started at http://127.24.159.193:46677/ using document root <none> and password file <none>
I20260925 11:01:57.958110 3138 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:57.958236 3138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:57.958498 3138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:57.961284 3138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "b1132ec1d8104be5a3ceee4c9de304e9"
format_stamp: "Formatted at 2026-09-25 11:01:57 on dist-test-slave-trms"
server_key: "2df73a33782530321e61b181e5f311ca"
server_key_iv: "67812a572923ecf093be9ce71474bf66"
server_key_version: "encryptionkey@0"
I20260925 11:01:57.962003 3138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "b1132ec1d8104be5a3ceee4c9de304e9"
format_stamp: "Formatted at 2026-09-25 11:01:57 on dist-test-slave-trms"
server_key: "2df73a33782530321e61b181e5f311ca"
server_key_iv: "67812a572923ecf093be9ce71474bf66"
server_key_version: "encryptionkey@0"
I20260925 11:01:57.967181 3138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 11:01:57.970621 3153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:57.972023 3138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 11:01:57.972168 3138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "b1132ec1d8104be5a3ceee4c9de304e9"
format_stamp: "Formatted at 2026-09-25 11:01:57 on dist-test-slave-trms"
server_key: "2df73a33782530321e61b181e5f311ca"
server_key_iv: "67812a572923ecf093be9ce71474bf66"
server_key_version: "encryptionkey@0"
I20260925 11:01:57.972345 3138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:57.984045 3138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:57.988253 3138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:57.988489 3138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:57.989356 3138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:57.990708 3138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:57.990777 3138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:57.990863 3138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:57.990926 3138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:58.019766 3138 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:43743
I20260925 11:01:58.019799 3265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:43743 every 8 connection(s)
I20260925 11:01:58.020998 3138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:01:58.025130 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 3138
I20260925 11:01:58.025337 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:01:58.026026 25215 external_mini_cluster.cc:1428] Setting key 07dd1019520f1a18344b9babcfd93be0
Sep 25 11:01:58 dist-test-slave-trms krb5kdc[3032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334117, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:58.045701 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:58.023798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:36685 (local address 127.24.159.254:37855)
0925 11:01:58.030555 (+ 6757us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:58.030562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:58.030582 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:01:58.030645 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:58.030651 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:58.030729 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:58.030847 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:58.032650 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:58.033352 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:58.035675 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:58.036076 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:58.036183 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:58.039781 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:58.039787 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:58.039791 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:58.039843 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:58.042688 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:58.043359 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:58.043364 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:58.043367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:58.043517 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:58.043898 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:58.043903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:58.043906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:58.044256 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:58.044397 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:58.045143 (+ 746us) server_negotiation.cc:299] Negotiation successful
0925 11:01:58.045355 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6625,"thread_start_us":6513,"threads_started":1}
I20260925 11:01:58.046622 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:58.024447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37855 (local address 127.24.159.193:36685)
0925 11:01:58.025519 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:58.025569 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:58.028468 (+ 2899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:58.031142 (+ 2674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:58.031152 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:58.031730 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:58.032487 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:58.032507 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:58.033501 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:58.033505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:58.035524 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:58.035545 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:58.036287 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:58.036294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:58.036542 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:58.037159 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:58.037181 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:58.039569 (+ 2388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:58.042821 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:58.042827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:58.042840 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:58.043241 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:58.043619 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:58.043623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:58.043625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:58.043792 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:58.044369 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:58.044375 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:58.044768 (+ 393us) client_negotiation.cc:769] Sending connection context
0925 11:01:58.044998 (+ 230us) client_negotiation.cc:240] Negotiation successful
0925 11:01:58.045393 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":392,"threads_started":1}
I20260925 11:01:58.048048 3266 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37855
I20260925 11:01:58.048323 3266 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:58.049041 3266 heartbeater.cc:511] Master 127.24.159.254:37855 requested a full tablet report, sending...
I20260925 11:01:58.051050 3077 ts_manager.cc:194] Registered new tserver with Master: b1132ec1d8104be5a3ceee4c9de304e9 (127.24.159.193:43743)
I20260925 11:01:58.052387 3077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:36685
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:01:58.078532 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37855
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:58.392112 3272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:58.392369 3272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:58.392464 3272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:58.399499 3272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:58.399600 3272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:58.399716 3272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:01:58.409181 3272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:58.410413 3272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:58.411640 3272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:58.424139 3278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:58.424243 3280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:58.424644 3277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:58.424955 3272 server_base.cc:1034] running on GCE node
I20260925 11:01:58.425552 3272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:58.426494 3272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:58.427680 3272 hybrid_clock.cc:648] HybridClock initialized: now 1790334118427634 us; error 35 us; skew 500 ppm
I20260925 11:01:58.427973 3272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:58 dist-test-slave-trms krb5kdc[3032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334118, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:58.434353 3272 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:01:58.435758 3272 webserver.cc:460] Webserver started at http://127.24.159.194:35225/ using document root <none> and password file <none>
I20260925 11:01:58.436398 3272 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:58.436499 3272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:58.436743 3272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:58.439666 3272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "f3ad908c84e34f009286e487f8c9ab8e"
format_stamp: "Formatted at 2026-09-25 11:01:58 on dist-test-slave-trms"
server_key: "dac46e77ecc34232b429ef8932ded06f"
server_key_iv: "16c2a18413aee29ca0da8c50997cbfba"
server_key_version: "encryptionkey@0"
I20260925 11:01:58.440400 3272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "f3ad908c84e34f009286e487f8c9ab8e"
format_stamp: "Formatted at 2026-09-25 11:01:58 on dist-test-slave-trms"
server_key: "dac46e77ecc34232b429ef8932ded06f"
server_key_iv: "16c2a18413aee29ca0da8c50997cbfba"
server_key_version: "encryptionkey@0"
I20260925 11:01:58.445286 3272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 11:01:58.448770 3287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:58.450142 3272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:01:58.450310 3272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "f3ad908c84e34f009286e487f8c9ab8e"
format_stamp: "Formatted at 2026-09-25 11:01:58 on dist-test-slave-trms"
server_key: "dac46e77ecc34232b429ef8932ded06f"
server_key_iv: "16c2a18413aee29ca0da8c50997cbfba"
server_key_version: "encryptionkey@0"
I20260925 11:01:58.450496 3272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:58.473217 3272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:58.477320 3272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:58.477589 3272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:58.478506 3272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:58.479780 3272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:58.479851 3272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:58.479961 3272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:58.480010 3272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:58.507905 3272 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:37841
I20260925 11:01:58.507963 3399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:37841 every 8 connection(s)
I20260925 11:01:58.509147 3272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:01:58.519251 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 3272
I20260925 11:01:58.519450 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:01:58.519943 25215 external_mini_cluster.cc:1428] Setting key f0ee445dc6e968189e03c5a318f4fa45
Sep 25 11:01:58 dist-test-slave-trms krb5kdc[3032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334118, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:58.528092 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:58.511615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:41097 (local address 127.24.159.254:37855)
0925 11:01:58.511807 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:58.511813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:58.512964 (+ 1151us) server_negotiation.cc:407] Connection header received
0925 11:01:58.514743 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:58.514748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:58.514815 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:58.514890 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:58.516249 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:58.516974 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:58.518464 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:58.518862 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:58.518949 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:58.522689 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:58.522695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:58.522699 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:58.522735 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:58.525234 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:58.525900 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:58.525904 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:58.525907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:58.526032 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:58.526464 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:58.526468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:58.526470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:58.526720 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:58.526857 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:58.527524 (+ 667us) server_negotiation.cc:299] Negotiation successful
0925 11:01:58.527752 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:01:58.529048 3402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:58.512118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37855 (local address 127.24.159.194:41097)
0925 11:01:58.512792 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:58.512847 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:58.514491 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:58.515079 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:58.515087 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:58.515581 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:58.516122 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:58.516139 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:58.517098 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:58.517102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:58.518363 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:58.518375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:58.519054 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:58.519061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:58.519329 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:58.519994 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:58.520023 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:58.522490 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:58.525370 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:58.525379 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:58.525396 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:58.525784 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:58.526131 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:58.526136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:58.526139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:58.526356 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:58.526845 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:58.526852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:58.527180 (+ 328us) client_negotiation.cc:769] Sending connection context
0925 11:01:58.527418 (+ 238us) client_negotiation.cc:240] Negotiation successful
0925 11:01:58.527921 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":254,"threads_started":1}
I20260925 11:01:58.530512 3400 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37855
I20260925 11:01:58.530781 3400 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:58.531438 3400 heartbeater.cc:511] Master 127.24.159.254:37855 requested a full tablet report, sending...
I20260925 11:01:58.532819 3077 ts_manager.cc:194] Registered new tserver with Master: f3ad908c84e34f009286e487f8c9ab8e (127.24.159.194:37841)
I20260925 11:01:58.533589 3077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:41097
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:01:58.565166 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37855
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:01:58.875295 3405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:01:58.875589 3405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:01:58.875689 3405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:01:58.882732 3405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:01:58.882840 3405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:01:58.882964 3405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:01:58.892520 3405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37855
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:01:58.893788 3405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:01:58.895041 3405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:01:58.907028 3410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:58.906984 3411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:01:58.907078 3413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:01:58.908444 3405 server_base.cc:1034] running on GCE node
I20260925 11:01:58.909000 3405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:01:58.909839 3405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:01:58.911036 3405 hybrid_clock.cc:648] HybridClock initialized: now 1790334118910986 us; error 42 us; skew 500 ppm
I20260925 11:01:58.911343 3405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:01:58 dist-test-slave-trms krb5kdc[3032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334118, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:01:58.914245 3405 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:01:58.915591 3405 webserver.cc:460] Webserver started at http://127.24.159.195:43691/ using document root <none> and password file <none>
I20260925 11:01:58.916275 3405 fs_manager.cc:359] Metadata directory not provided
I20260925 11:01:58.916386 3405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:01:58.916663 3405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:01:58.919543 3405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "253ea1140a2c400791601ea9ee9db312"
format_stamp: "Formatted at 2026-09-25 11:01:58 on dist-test-slave-trms"
server_key: "3c096c72d82236220116f775cb7acd7b"
server_key_iv: "8e14df07b6c77b7f9cc3c769f3b54fdf"
server_key_version: "encryptionkey@0"
I20260925 11:01:58.920310 3405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "253ea1140a2c400791601ea9ee9db312"
format_stamp: "Formatted at 2026-09-25 11:01:58 on dist-test-slave-trms"
server_key: "3c096c72d82236220116f775cb7acd7b"
server_key_iv: "8e14df07b6c77b7f9cc3c769f3b54fdf"
server_key_version: "encryptionkey@0"
I20260925 11:01:58.925292 3405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:01:58.928449 3420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:58.929765 3405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:01:58.929926 3405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "253ea1140a2c400791601ea9ee9db312"
format_stamp: "Formatted at 2026-09-25 11:01:58 on dist-test-slave-trms"
server_key: "3c096c72d82236220116f775cb7acd7b"
server_key_iv: "8e14df07b6c77b7f9cc3c769f3b54fdf"
server_key_version: "encryptionkey@0"
I20260925 11:01:58.930099 3405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:01:58.963523 3405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:01:58.967377 3405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:01:58.967648 3405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:01:58.968536 3405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:01:58.969813 3405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:01:58.969900 3405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:58.970017 3405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:01:58.970068 3405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:01:58.998584 3405 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:37533
I20260925 11:01:58.998628 3532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:37533 every 8 connection(s)
I20260925 11:01:58.999796 3405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:01:59.005069 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 3405
I20260925 11:01:59.005240 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:01:59.005614 25215 external_mini_cluster.cc:1428] Setting key 16234658f2081c082b3cdd5fe150e751
Sep 25 11:01:59 dist-test-slave-trms krb5kdc[3032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334118, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:01:59.018020 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.002136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:42809 (local address 127.24.159.254:37855)
0925 11:01:59.002349 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.002355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.003561 (+ 1206us) server_negotiation.cc:407] Connection header received
0925 11:01:59.005256 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.005263 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.005351 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.005440 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:59.007011 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.007749 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.009279 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.009670 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.009774 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.012934 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:59.012938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:59.012942 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:59.012973 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:59.015216 (+ 2243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.015885 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.015889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.015891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.016020 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.016408 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.016411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.016413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.016670 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:59.016797 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.017453 (+ 656us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.017676 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260925 11:01:59.018947 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.002692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37855 (local address 127.24.159.195:42809)
0925 11:01:59.003386 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:59.003442 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:59.004972 (+ 1530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:59.005702 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:59.005713 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:59.006281 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:01:59.006844 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.006861 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.007898 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.007903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.009146 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.009157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.009873 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.009880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.010109 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.010741 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:01:59.010762 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:01:59.012734 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:01:59.015347 (+ 2613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:59.015355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:59.015371 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:59.015767 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:59.016123 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:01:59.016127 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:01:59.016131 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:01:59.016298 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:01:59.016788 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:01:59.016796 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:01:59.017105 (+ 309us) client_negotiation.cc:769] Sending connection context
0925 11:01:59.017339 (+ 234us) client_negotiation.cc:240] Negotiation successful
0925 11:01:59.017843 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":288,"threads_started":1}
I20260925 11:01:59.021195 3533 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37855
I20260925 11:01:59.021478 3533 heartbeater.cc:464] Registering TS with master...
I20260925 11:01:59.022135 3533 heartbeater.cc:511] Master 127.24.159.254:37855 requested a full tablet report, sending...
I20260925 11:01:59.023473 3077 ts_manager.cc:194] Registered new tserver with Master: 253ea1140a2c400791601ea9ee9db312 (127.24.159.195:37533)
I20260925 11:01:59.024264 3077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:42809
I20260925 11:01:59.032351 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:01:59.047957 3268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.036636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40442 (local address 127.24.159.254:37855)
0925 11:01:59.036790 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.036796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.037202 (+ 406us) server_negotiation.cc:407] Connection header received
0925 11:01:59.037358 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.037361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.037421 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.037501 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:59.038888 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.039622 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.040814 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.041227 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.041332 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.043407 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:59.043412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:59.043415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:59.043448 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:59.045514 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.046090 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.046095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.046098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.046273 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.046613 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.046618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.046622 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.046939 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:59.047146 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.047464 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.047663 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 11:01:59.055080 3266 heartbeater.cc:503] Master 127.24.159.254:37855 was elected leader, sending a full tablet report...
I20260925 11:01:59.051704 3077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:01:59.058617 3077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:01:59.090183 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.069314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34730 (local address 127.24.159.194:37841)
0925 11:01:59.082719 (+ 13405us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.082728 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.082754 (+ 26us) server_negotiation.cc:407] Connection header received
0925 11:01:59.082867 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.082876 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.083165 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.083384 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:59.083931 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.085268 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.087522 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.088966 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.089130 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.089297 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.089642 (+ 345us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.089858 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13244,"thread_start_us":14638,"threads_started":1}
I20260925 11:01:59.090220 3548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.069163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:37841 (local address 127.0.0.1:34730)
0925 11:01:59.080386 (+ 11223us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:59.080439 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:59.080624 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:59.083385 (+ 2761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:59.083390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:59.083416 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:59.083763 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.083772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.085405 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.085410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.087310 (+ 1900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.087328 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.089152 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.089158 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.089387 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.089449 (+ 62us) client_negotiation.cc:769] Sending connection context
0925 11:01:59.089578 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 11:01:59.089762 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10947,"mutex_wait_us":23,"thread_start_us":12702,"threads_started":1}
I20260925 11:01:59.091590 3550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.070356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.24.159.195:37533)
0925 11:01:59.079295 (+ 8939us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.079306 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.084159 (+ 4853us) server_negotiation.cc:407] Connection header received
0925 11:01:59.084297 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.084304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.084538 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.084714 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:59.085382 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.087024 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.088960 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.090591 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.090762 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.090929 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.091116 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.091270 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8741,"thread_start_us":8574,"threads_started":1}
I20260925 11:01:59.091579 3547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.069698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:37533 (local address 127.0.0.1:34386)
0925 11:01:59.084037 (+ 14339us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:59.084054 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:59.084169 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:59.084725 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:59.084729 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:59.084753 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:59.085178 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.085194 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.087192 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.087196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.088823 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.088835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.090762 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.090768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.090966 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.090990 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 11:01:59.091064 (+ 74us) client_negotiation.cc:240] Negotiation successful
0925 11:01:59.091179 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14235,"mutex_wait_us":45,"thread_start_us":10288,"threads_started":1}
I20260925 11:01:59.092320 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.069403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43743 (local address 127.0.0.1:45268)
0925 11:01:59.082164 (+ 12761us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:59.082183 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:59.084207 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:59.084963 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:59.084967 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:59.084994 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:59.085435 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.085446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.087402 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.087407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.089507 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.089518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.091550 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.091557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.091725 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.091753 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 11:01:59.091840 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 11:01:59.091969 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12613,"mutex_wait_us":7,"thread_start_us":14348,"threads_started":1}
I20260925 11:01:59.093514 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.070034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.24.159.193:43743)
0925 11:01:59.081151 (+ 11117us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.081162 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.082299 (+ 1137us) server_negotiation.cc:407] Connection header received
0925 11:01:59.084399 (+ 2100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.084405 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.084586 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.084740 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:59.085608 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.087292 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.089671 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.091070 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.091244 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.091399 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.092599 (+ 1200us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.093209 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10942,"thread_start_us":10796,"threads_started":1}
I20260925 11:01:59.094892 3468 tablet_service.cc:1467] Processing CreateTablet for tablet 4b84a932aa9b45a6bb357eff089062a9 (DEFAULT_TABLE table=test-table [id=d985a411674b4da48e3bdbfe737fdace]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:59.094731 3335 tablet_service.cc:1467] Processing CreateTablet for tablet 4b84a932aa9b45a6bb357eff089062a9 (DEFAULT_TABLE table=test-table [id=d985a411674b4da48e3bdbfe737fdace]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:59.096020 3468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b84a932aa9b45a6bb357eff089062a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:59.096091 3335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b84a932aa9b45a6bb357eff089062a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:59.096836 3201 tablet_service.cc:1467] Processing CreateTablet for tablet 4b84a932aa9b45a6bb357eff089062a9 (DEFAULT_TABLE table=test-table [id=d985a411674b4da48e3bdbfe737fdace]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:01:59.098132 3201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b84a932aa9b45a6bb357eff089062a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:01:59.106782 3551 tablet_bootstrap.cc:492] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: Bootstrap starting.
I20260925 11:01:59.110059 3551 tablet_bootstrap.cc:654] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:59.111016 3551 log.cc:824] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:59.115083 3554 tablet_bootstrap.cc:492] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: Bootstrap starting.
I20260925 11:01:59.116369 3552 tablet_bootstrap.cc:492] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: Bootstrap starting.
I20260925 11:01:59.116971 3551 tablet_bootstrap.cc:492] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: No bootstrap required, opened a new log
I20260925 11:01:59.117229 3551 ts_tablet_manager.cc:1397] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260925 11:01:59.118811 3554 tablet_bootstrap.cc:654] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:59.119588 3552 tablet_bootstrap.cc:654] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: Neither blocks nor log segments found. Creating new log.
I20260925 11:01:59.120016 3554 log.cc:824] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:59.120674 3552 log.cc:824] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: Log is configured to *not* fsync() on all Append() calls
I20260925 11:01:59.122660 3554 tablet_bootstrap.cc:492] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: No bootstrap required, opened a new log
I20260925 11:01:59.122886 3554 ts_tablet_manager.cc:1397] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 11:01:59.123430 3552 tablet_bootstrap.cc:492] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: No bootstrap required, opened a new log
I20260925 11:01:59.123665 3552 ts_tablet_manager.cc:1397] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 11:01:59.127054 3551 raft_consensus.cc:348] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } }
I20260925 11:01:59.127487 3551 raft_consensus.cc:374] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:59.127583 3551 raft_consensus.cc:729] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 253ea1140a2c400791601ea9ee9db312, State: Initialized, Role: FOLLOWER
I20260925 11:01:59.128181 3551 consensus_queue.cc:260] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } }
I20260925 11:01:59.129253 3533 heartbeater.cc:503] Master 127.24.159.254:37855 was elected leader, sending a full tablet report...
I20260925 11:01:59.129851 3554 raft_consensus.cc:348] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } }
I20260925 11:01:59.130158 3554 raft_consensus.cc:374] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:59.130322 3554 raft_consensus.cc:729] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1132ec1d8104be5a3ceee4c9de304e9, State: Initialized, Role: FOLLOWER
I20260925 11:01:59.131055 3551 ts_tablet_manager.cc:1428] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
I20260925 11:01:59.131027 3554 consensus_queue.cc:260] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } }
I20260925 11:01:59.133040 3552 raft_consensus.cc:348] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } }
I20260925 11:01:59.133512 3552 raft_consensus.cc:374] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:01:59.133613 3552 raft_consensus.cc:729] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ad908c84e34f009286e487f8c9ab8e, State: Initialized, Role: FOLLOWER
I20260925 11:01:59.134287 3552 consensus_queue.cc:260] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } }
I20260925 11:01:59.135465 3400 heartbeater.cc:503] Master 127.24.159.254:37855 was elected leader, sending a full tablet report...
I20260925 11:01:59.135556 3554 ts_tablet_manager.cc:1428] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260925 11:01:59.136922 3552 ts_tablet_manager.cc:1428] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260925 11:01:59.182785 3559 raft_consensus.cc:482] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:01:59.183033 3559 raft_consensus.cc:504] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } }
I20260925 11:01:59.184553 3559 leader_election.cc:288] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1132ec1d8104be5a3ceee4c9de304e9 (127.24.159.193:43743), 253ea1140a2c400791601ea9ee9db312 (127.24.159.195:37533)
I20260925 11:01:59.192881 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.184976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:35975 (local address 127.24.159.193:43743)
0925 11:01:59.185186 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.185193 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.185859 (+ 666us) server_negotiation.cc:407] Connection header received
0925 11:01:59.186082 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.186088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.186154 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.186273 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:59.186866 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.187984 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.190774 (+ 2790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.191966 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.192114 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.192185 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.192531 (+ 346us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.192656 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 11:01:59.193009 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.184740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43743 (local address 127.24.159.194:35975)
0925 11:01:59.185742 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:59.185760 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:59.185913 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:59.186292 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:59.186296 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:59.186319 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:59.186734 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.186747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.188113 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.188117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.190616 (+ 2499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.190625 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.192194 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.192201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.192367 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.192404 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 11:01:59.192493 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 11:01:59.192624 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":832,"mutex_wait_us":85,"spinlock_wait_cycles":29440,"thread_start_us":891,"threads_started":1}
I20260925 11:01:59.193053 3550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.185285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:40405 (local address 127.24.159.195:37533)
0925 11:01:59.185480 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.185486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.185979 (+ 493us) server_negotiation.cc:407] Connection header received
0925 11:01:59.186122 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.186126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.186181 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.186269 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:01:59.187060 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.188116 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.190758 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.191966 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.192114 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.192185 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.192686 (+ 501us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.192810 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:01:59.193564 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.185009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:37533 (local address 127.24.159.194:40405)
0925 11:01:59.185884 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0925 11:01:59.185897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 11:01:59.186009 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:01:59.186293 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:01:59.186298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:01:59.186384 (+ 86us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:01:59.186836 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.186848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.188228 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.188233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.190608 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:01:59.190624 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.192325 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.192330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:01:59.192509 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.192539 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 11:01:59.193216 (+ 677us) client_negotiation.cc:240] Negotiation successful
0925 11:01:59.193312 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":70,"spinlock_wait_cycles":25344,"thread_start_us":356,"threads_started":1}
I20260925 11:01:59.194109 3221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b84a932aa9b45a6bb357eff089062a9" candidate_uuid: "f3ad908c84e34f009286e487f8c9ab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" is_pre_election: true
I20260925 11:01:59.194499 3488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b84a932aa9b45a6bb357eff089062a9" candidate_uuid: "f3ad908c84e34f009286e487f8c9ab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "253ea1140a2c400791601ea9ee9db312" is_pre_election: true
I20260925 11:01:59.194746 3221 raft_consensus.cc:2454] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3ad908c84e34f009286e487f8c9ab8e in term 0.
I20260925 11:01:59.195022 3488 raft_consensus.cc:2454] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3ad908c84e34f009286e487f8c9ab8e in term 0.
I20260925 11:01:59.195421 3290 leader_election.cc:302] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1132ec1d8104be5a3ceee4c9de304e9, f3ad908c84e34f009286e487f8c9ab8e; no voters:
I20260925 11:01:59.195785 3559 raft_consensus.cc:2781] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:01:59.195916 3559 raft_consensus.cc:482] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:01:59.196017 3559 raft_consensus.cc:3037] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:59.198094 3559 raft_consensus.cc:504] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } }
I20260925 11:01:59.198676 3559 leader_election.cc:288] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [CANDIDATE]: Term 1 election: Requested vote from peers b1132ec1d8104be5a3ceee4c9de304e9 (127.24.159.193:43743), 253ea1140a2c400791601ea9ee9db312 (127.24.159.195:37533)
I20260925 11:01:59.199054 3221 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b84a932aa9b45a6bb357eff089062a9" candidate_uuid: "f3ad908c84e34f009286e487f8c9ab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1132ec1d8104be5a3ceee4c9de304e9"
I20260925 11:01:59.199137 3488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b84a932aa9b45a6bb357eff089062a9" candidate_uuid: "f3ad908c84e34f009286e487f8c9ab8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "253ea1140a2c400791601ea9ee9db312"
I20260925 11:01:59.199257 3221 raft_consensus.cc:3037] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:59.199316 3488 raft_consensus.cc:3037] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:01:59.202346 3488 raft_consensus.cc:2454] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3ad908c84e34f009286e487f8c9ab8e in term 1.
I20260925 11:01:59.202346 3221 raft_consensus.cc:2454] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3ad908c84e34f009286e487f8c9ab8e in term 1.
I20260925 11:01:59.202705 3289 leader_election.cc:302] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 253ea1140a2c400791601ea9ee9db312, f3ad908c84e34f009286e487f8c9ab8e; no voters:
I20260925 11:01:59.203040 3559 raft_consensus.cc:2781] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:01:59.203444 3559 raft_consensus.cc:686] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 1 LEADER]: Becoming Leader. State: Replica: f3ad908c84e34f009286e487f8c9ab8e, State: Running, Role: LEADER
I20260925 11:01:59.203856 3559 consensus_queue.cc:237] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } }
I20260925 11:01:59.208214 3076 catalog_manager.cc:5495] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e reported cstate change: term changed from 0 to 1, leader changed from <none> to f3ad908c84e34f009286e487f8c9ab8e (127.24.159.194). New cstate: current_term: 1 leader_uuid: "f3ad908c84e34f009286e487f8c9ab8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f3ad908c84e34f009286e487f8c9ab8e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 37841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 } health_report { overall_health: UNKNOWN } } }
I20260925 11:01:59.233536 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.228812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.24.159.194:37841)
0925 11:01:59.229022 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.229027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.229046 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:01:59.229191 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.229195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.229272 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.229393 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:01:59.229674 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.230422 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.231607 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.231992 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.232082 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.232491 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:01:59.233008 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:01:59.233110 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.233180 (+ 70us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.233244 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:01:59.241639 3221 raft_consensus.cc:1261] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [term 1 FOLLOWER]: Refusing update from remote peer f3ad908c84e34f009286e487f8c9ab8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:59.241639 3488 raft_consensus.cc:1261] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [term 1 FOLLOWER]: Refusing update from remote peer f3ad908c84e34f009286e487f8c9ab8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:01:59.242547 3562 consensus_queue.cc:1035] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "253ea1140a2c400791601ea9ee9db312" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:01:59.242880 3559 consensus_queue.cc:1035] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1132ec1d8104be5a3ceee4c9de304e9" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 11:01:59.251338 3534 tablet.cc:2369] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:01:59.253756 3565 mvcc.cc:205] Tried to move back new op lower bound from 7333208552404148224 to 7333208552262356992. Current Snapshot: MvccSnapshot[applied={T|T < 7333208552404148224}]
I20260925 11:01:59.260172 3567 mvcc.cc:205] Tried to move back new op lower bound from 7333208552404148224 to 7333208552262356992. Current Snapshot: MvccSnapshot[applied={T|T < 7333208552404148224}]
I20260925 11:01:59.260501 3566 mvcc.cc:205] Tried to move back new op lower bound from 7333208552404148224 to 7333208552262356992. Current Snapshot: MvccSnapshot[applied={T|T < 7333208552404148224}]
I20260925 11:01:59.268627 3076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:59.268894 3076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:01:59.272318 3076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffc28f6f85014dbebed9e5195a4550b7: Sending DeleteTablet for 3 replicas of tablet 4b84a932aa9b45a6bb357eff089062a9
I20260925 11:01:59.273389 3201 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b84a932aa9b45a6bb357eff089062a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:59 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:45268
I20260925 11:01:59.273412 3335 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b84a932aa9b45a6bb357eff089062a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:59 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:34730
I20260925 11:01:59.273553 3468 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b84a932aa9b45a6bb357eff089062a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:01:59 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:34386
I20260925 11:01:59.274344 3574 tablet_replica.cc:330] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: stopping tablet replica
I20260925 11:01:59.274744 3574 raft_consensus.cc:2229] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:01:59.275184 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 3138
I20260925 11:01:59.275410 3574 raft_consensus.cc:2258] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:59.277501 3574 ts_tablet_manager.cc:1905] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:59.284803 3574 ts_tablet_manager.cc:1918] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:59.285053 3574 log.cc:1196] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/4b84a932aa9b45a6bb357eff089062a9
I20260925 11:01:59.285645 3574 ts_tablet_manager.cc:1939] T 4b84a932aa9b45a6bb357eff089062a9 P f3ad908c84e34f009286e487f8c9ab8e: Deleting consensus metadata
I20260925 11:01:59.286688 3575 tablet_replica.cc:330] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: stopping tablet replica
I20260925 11:01:59.287214 3575 raft_consensus.cc:2229] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:59.287599 3061 catalog_manager.cc:4841] TS f3ad908c84e34f009286e487f8c9ab8e (127.24.159.194:37841): tablet 4b84a932aa9b45a6bb357eff089062a9 (table test-table [id=d985a411674b4da48e3bdbfe737fdace]) successfully deleted
I20260925 11:01:59.287750 3575 raft_consensus.cc:2258] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:59.289391 3576 tablet_replica.cc:330] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: stopping tablet replica
I20260925 11:01:59.289796 3576 raft_consensus.cc:2229] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:01:59.289903 3575 ts_tablet_manager.cc:1905] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:59.290298 3576 raft_consensus.cc:2258] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:01:59.292368 3576 ts_tablet_manager.cc:1905] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:01:59.296779 3575 ts_tablet_manager.cc:1918] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:59.296993 3575 log.cc:1196] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/4b84a932aa9b45a6bb357eff089062a9
I20260925 11:01:59.297550 3575 ts_tablet_manager.cc:1939] T 4b84a932aa9b45a6bb357eff089062a9 P b1132ec1d8104be5a3ceee4c9de304e9: Deleting consensus metadata
Sep 25 11:01:59 dist-test-slave-trms krb5kdc[3032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:01:59.299119 3576 ts_tablet_manager.cc:1918] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:01:59.299306 3576 log.cc:1196] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/4b84a932aa9b45a6bb357eff089062a9
I20260925 11:01:59.299608 3063 catalog_manager.cc:4841] TS b1132ec1d8104be5a3ceee4c9de304e9 (127.24.159.193:43743): tablet 4b84a932aa9b45a6bb357eff089062a9 (table test-table [id=d985a411674b4da48e3bdbfe737fdace]) successfully deleted
I20260925 11:01:59.300068 3576 ts_tablet_manager.cc:1939] T 4b84a932aa9b45a6bb357eff089062a9 P 253ea1140a2c400791601ea9ee9db312: Deleting consensus metadata
I20260925 11:01:59.302079 3062 catalog_manager.cc:4841] TS 253ea1140a2c400791601ea9ee9db312 (127.24.159.195:37533): tablet 4b84a932aa9b45a6bb357eff089062a9 (table test-table [id=d985a411674b4da48e3bdbfe737fdace]) successfully deleted
I20260925 11:01:59.305191 3549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.278518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45280 (local address 127.24.159.193:43743)
0925 11:01:59.278707 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.278714 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.288729 (+ 10015us) server_negotiation.cc:407] Connection header received
0925 11:01:59.288943 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.288949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.289050 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.289154 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:59.290637 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.291711 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.293405 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.294015 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.294127 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.299039 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:59.299054 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:59.299072 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:59.299113 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:59.302677 (+ 3564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.303466 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.303473 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.303475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.303586 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.303940 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.303942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.303944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.304268 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:59.304494 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.304799 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.304949 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:01:59.305894 3261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:59.391796 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 3138
I20260925 11:01:59.405424 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 3272
Sep 25 11:01:59 dist-test-slave-trms krb5kdc[3032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:01:59.420147 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.406159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.24.159.194:37841)
0925 11:01:59.406382 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.406388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.406406 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:01:59.406527 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.406533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.406613 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.406698 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:59.407801 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.408627 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.409846 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.410329 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.410448 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.414221 (+ 3773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:59.414240 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:59.414260 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:59.414306 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:59.417234 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.417988 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.417998 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.418002 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.418219 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.418593 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.418597 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.418601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.419047 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:59.419356 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.419531 (+ 175us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.419763 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260925 11:01:59.420801 3395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:59.530017 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 3272
I20260925 11:01:59.547097 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 3405
Sep 25 11:01:59 dist-test-slave-trms krb5kdc[3032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:01:59.560663 3550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:01:59.547790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34396 (local address 127.24.159.195:37533)
0925 11:01:59.547969 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 11:01:59.547975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:01:59.547993 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:01:59.548138 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:01:59.548142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:01:59.548208 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:01:59.548287 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:01:59.549491 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.550220 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.551391 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:01:59.551774 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:01:59.551878 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:01:59.555834 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:01:59.555848 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:01:59.555865 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:01:59.555895 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:01:59.558220 (+ 2325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.558815 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.558822 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.558824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.558954 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:01:59.559275 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:01:59.559280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:01:59.559284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:01:59.559598 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:01:59.559872 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0925 11:01:59.560175 (+ 303us) server_negotiation.cc:299] Negotiation successful
0925 11:01:59.560381 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 11:01:59.561259 3528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:59.646868 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 3405
I20260925 11:01:59.660089 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 3045
I20260925 11:01:59.661144 3107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:01:59.768559 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 3045
2026-09-25T11:01:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (3965 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:01:59 dist-test-slave-trms krb5kdc[3584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:01:59 dist-test-slave-trms krb5kdc[3584](info): set up 2 sockets
Sep 25 11:01:59 dist-test-slave-trms krb5kdc[3584](info): commencing operation
krb5kdc: starting...
W20260925 11:02:00.771098 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:02:00 dist-test-slave-trms krb5kdc[3584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:02:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:02:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.159.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.159.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/rangeradmin_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/rangeradmin_127.24.159.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.159.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.159.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/rangerlookup_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/rangerlookup_127.24.159.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.159.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.159.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/HTTP_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/HTTP_127.24.159.212@KRBTEST.COM.keytab.
I20260925 11:02:01.067139 25215 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskw_hhNs/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/postgres -l logfile start
2026-09-25 11:02:03.394 UTC [3614] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 11:02:03.394 UTC [3614] LOG: listening on IPv4 address "127.24.159.212", port 49175
2026-09-25 11:02:03.397 UTC [3614] LOG: listening on Unix socket "/tmp/.s.PGSQL.49175"
2026-09-25 11:02:03.417 UTC [3617] LOG: database system was shut down at 2026-09-25 11:02:02 UTC
2026-09-25 11:02:03.421 UTC [3614] LOG: database system is ready to accept connections
I20260925 11:02:04.318779 25215 mini_postgres.cc:96] Postgres bound to 49175
2026-09-25 11:02:04.335 UTC [3625] FATAL: database "slave" does not exist
127.24.159.212:49175 - accepting connections
I20260925 11:02:04.335996 25215 mini_ranger.cc:161] Starting Ranger...
I20260925 11:02:04.361032 25215 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 11:02:04.866137 25215 mini_ranger.cc:87] Created ranger Postgres database
I20260925 11:02:04.866281 25215 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-admin
2026-09-25 11:02:05,147 [I] DB FLAVOR :POSTGRES
2026-09-25 11:02:05,148 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 11:02:05,148 [I] Checking connection..
2026-09-25 11:02:05,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 11:02:05,399 [I] Checking connection passed.
2026-09-25 11:02:05,400 [I] --------- Verifying version history table ---------
2026-09-25 11:02:05,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 11:02:05,661 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 11:02:05,662 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 11:02:05,894 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 11:02:05,895 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 11:02:05,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 11:02:06.139 UTC [3689] WARNING: there is no transaction in progress
2026-09-25 11:02:06,145 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 11:02:06,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 11:02:06,396 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 11:02:06,396 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 11:02:06,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 11:02:06,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 11:02:06,868 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-trms.c.gcp-upstream.internal','N') ;"
2026-09-25 11:02:07,111 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 11:02:07,342 [I] Table x_portal_user does not exist in database ranger
2026-09-25 11:02:07,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 11:02:07,574 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 11:02:07,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 11:02:07,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 11:02:08,070 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 11:02:08,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 11:02:09.000 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.016 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.031 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.045 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.062 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.078 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.130 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.136 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.149 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.159 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.168 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.181 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.190 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.196 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.202 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09.208 UTC [3797] WARNING: there is no transaction in progress
2026-09-25 11:02:09,822 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 11:02:09,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 11:02:10,068 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 11:02:10,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 11:02:10,321 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 11:02:10,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 11:02:10,561 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 11:02:10,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 11:02:10,800 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 11:02:11,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-trms.c.gcp-upstream.internal';"
2026-09-25 11:02:11,268 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 11:02:11,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 11:02:11,508 [I] DB_PATCHES have already been applied
I20260925 11:02:11.513638 25215 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-admin:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskw_hhNs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 11:02:11.513727 25215 mini_ranger.cc:193] Using host: 127.24.159.212
I20260925 11:02:11.527629 25215 mini_ranger.cc:227] Ranger admin URL: http://127.24.159.212:53881
Sep 25, 2026 11:02:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 11:02:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 11:02:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 11:02:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 11:02:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 11:02:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 11:02:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 11:02:12.525686 25215 mini_ranger.cc:160] Time spent starting Ranger: real 8.190s user 0.003s sys 0.036s
Sep 25, 2026 11:02:31 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 11:02:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 25, 2026 11:02:31 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 11:02:31 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 11:02:32 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 11:02:32.584654 25215 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.159.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.159.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/rangerkms_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/rangerkms_127.24.159.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.159.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.159.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.159.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/HTTP_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.159.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/HTTP_127.24.159.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 11:02:32 dist-test-slave-trms krb5kdc[3584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334152, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 11:02:32.748525 25215 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 11:02:32.768124 25215 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 11:02:33.528663 25215 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 11:02:33.528805 25215 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-kms
I20260925 11:02:33.528925 25215 mini_ranger_kms.cc:227] Using postgres at 127.24.159.212:49175
2026-09-25 11:02:33,921 [I] DB FLAVOR :POSTGRES
2026-09-25 11:02:33,921 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 11:02:33,921 [I] Checking connection
2026-09-25 11:02:33,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 11:02:34,175 [I] connection success
2026-09-25 11:02:34,175 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 11:02:34,176 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 11:02:34,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 11:02:34,401 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 11:02:34,401 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 11:02:34,401 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 11:02:34,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:49175/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 11:02:34,645 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 11:02:35.089854 25215 mini_ranger_kms.cc:310] Created kms service
I20260925 11:02:35.258445 25215 mini_ranger_kms.cc:326] Created kudu user
I20260925 11:02:35.298569 25215 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 11:02:35 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334152, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:02:35.687112 25215 mini_ranger_kms.cc:384] Added ranger policy
I20260925 11:02:35.687413 25215 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-kms:/tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 11:02:35.687466 25215 mini_ranger_kms.cc:240] Using host: 127.24.159.212
I20260925 11:02:35.708276 25215 mini_ranger_kms.cc:276] Ranger KMS PID: 3977
I20260925 11:02:35.708467 25215 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.159.212:59283
Sep 25, 2026 11:02:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 11:02:36 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 11:02:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 11:02:36 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 11:02:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 11:02:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 25 11:02:37 dist-test-slave-trms krb5kdc[3584](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790334157, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.159.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:02:37.777968 25215 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.029s user 0.126s sys 0.285s
I20260925 11:02:37.778265 25215 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 11:02:37.778508 25215 mini_ranger_kms.cc:401] 127.24.159.212:59283/kms/v1/keys
Sep 25 11:02:38 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334157, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.159.212@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
Sep 25 11:02:38 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334157, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.159.212@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
Sep 25 11:02:40 dist-test-slave-trms krb5kdc[3584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:02:40.788127 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:33813
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:33813
--ranger_config_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:59283/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:02:41.110064 4028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:02:41.110366 4028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:02:41.110494 4028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:02:41.117659 4028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:02:41.117818 4028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:02:41.117877 4028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:02:41.117949 4028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:02:41.118000 4028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:02:41.127831 4028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.159.212:59283/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:33813
--ranger_config_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:33813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:02:41.129086 4028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:02:41.130370 4028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:02:41.141696 4036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:02:41.141764 4034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:02:41.141791 4033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:02:41.142705 4028 server_base.cc:1034] running on GCE node
I20260925 11:02:41.143473 4028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:02:41.144867 4028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:02:41.146077 4028 hybrid_clock.cc:648] HybridClock initialized: now 1790334161146033 us; error 32 us; skew 500 ppm
I20260925 11:02:41.146435 4028 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:02:41 dist-test-slave-trms krb5kdc[3584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334161, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:02:41.152580 4028 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:02:41.153996 4028 webserver.cc:460] Webserver started at http://127.24.159.254:38859/ using document root <none> and password file <none>
I20260925 11:02:41.154646 4028 fs_manager.cc:359] Metadata directory not provided
I20260925 11:02:41.154752 4028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:02:41.155045 4028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 11:02:41 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334161, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:02:41.203773 4028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "9aeddca43fc849c490342bf985f75f1b"
format_stamp: "Formatted at 2026-09-25 11:02:41 on dist-test-slave-trms"
server_key: "4b58cb4c786632db7e7b710b5e9a1600"
server_key_iv: "852ee0512efdf21feaac83f9fe8b9d93"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:41.204607 4028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "9aeddca43fc849c490342bf985f75f1b"
format_stamp: "Formatted at 2026-09-25 11:02:41 on dist-test-slave-trms"
server_key: "4b58cb4c786632db7e7b710b5e9a1600"
server_key_iv: "852ee0512efdf21feaac83f9fe8b9d93"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:41.210000 4028 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:02:41.227130 4043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:41.229079 4028 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 11:02:41.229288 4028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "9aeddca43fc849c490342bf985f75f1b"
format_stamp: "Formatted at 2026-09-25 11:02:41 on dist-test-slave-trms"
server_key: "4b58cb4c786632db7e7b710b5e9a1600"
server_key_iv: "852ee0512efdf21feaac83f9fe8b9d93"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:41.229562 4028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:02:41.254798 4028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:02:41.259438 4028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:02:41.259745 4028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:02:41.277741 4094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:33813 every 8 connection(s)
I20260925 11:02:41.277750 4028 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:33813
I20260925 11:02:41.279496 4028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:02:41.283360 4095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:02:41.286417 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 4028
I20260925 11:02:41.286630 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:02:41.287372 25215 external_mini_cluster.cc:1428] Setting key 6172e166524c18f154515b2174b03c2a
I20260925 11:02:41.296120 4095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b: Bootstrap starting.
I20260925 11:02:41.299670 4095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b: Neither blocks nor log segments found. Creating new log.
I20260925 11:02:41.300870 4095 log.cc:824] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b: Log is configured to *not* fsync() on all Append() calls
Sep 25 11:02:41 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:02:41.303692 4095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b: No bootstrap required, opened a new log
I20260925 11:02:41.309027 4098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:41.289489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36168 (local address 127.24.159.254:33813)
0925 11:02:41.290414 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:41.290431 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:41.290717 (+ 286us) server_negotiation.cc:407] Connection header received
0925 11:02:41.292397 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:41.292419 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:41.292817 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:41.293142 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:02:41.294556 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:41.295825 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:41.297427 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:41.298075 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:41.298232 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:41.302499 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:02:41.302507 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:02:41.302521 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:02:41.302556 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:02:41.305315 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:41.305947 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:41.305953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:41.305959 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:41.306115 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:41.306487 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:41.306490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:41.306492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:41.306980 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:02:41.307242 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:41.307670 (+ 428us) server_negotiation.cc:299] Negotiation successful
0925 11:02:41.308031 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":377,"threads_started":1}
I20260925 11:02:41.311453 4095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aeddca43fc849c490342bf985f75f1b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 33813 } }
I20260925 11:02:41.311853 4095 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:02:41.311949 4095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aeddca43fc849c490342bf985f75f1b, State: Initialized, Role: FOLLOWER
I20260925 11:02:41.312479 4095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aeddca43fc849c490342bf985f75f1b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 33813 } }
I20260925 11:02:41.312691 4095 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:02:41.312809 4095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:02:41.312944 4095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:02:41.315210 4095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aeddca43fc849c490342bf985f75f1b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 33813 } }
I20260925 11:02:41.315681 4095 leader_election.cc:302] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9aeddca43fc849c490342bf985f75f1b; no voters:
I20260925 11:02:41.316169 4095 leader_election.cc:288] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:02:41.316269 4100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:02:41.316851 4100 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [term 1 LEADER]: Becoming Leader. State: Replica: 9aeddca43fc849c490342bf985f75f1b, State: Running, Role: LEADER
I20260925 11:02:41.317490 4100 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aeddca43fc849c490342bf985f75f1b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 33813 } }
I20260925 11:02:41.318252 4095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:02:41.320259 4102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9aeddca43fc849c490342bf985f75f1b. Latest consensus state: current_term: 1 leader_uuid: "9aeddca43fc849c490342bf985f75f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aeddca43fc849c490342bf985f75f1b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 33813 } } }
I20260925 11:02:41.320806 4102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [sys.catalog]: This master's current role is: LEADER
I20260925 11:02:41.320243 4101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9aeddca43fc849c490342bf985f75f1b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9aeddca43fc849c490342bf985f75f1b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 33813 } } }
I20260925 11:02:41.321213 4101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b [sys.catalog]: This master's current role is: LEADER
I20260925 11:02:41.322564 4108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:02:41.327239 4108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:02:41.330507 4095 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260925 11:02:41.344905 4108 catalog_manager.cc:1347] Generated new cluster ID: 2eab46af2f5446bf8ce4fe2d9c436313
I20260925 11:02:41.345036 4108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:02:41.362954 4108 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:02:41.364288 4108 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:02:41.379686 4108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b: Generated new TSK 0
I20260925 11:02:41.380599 4108 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:02:42.093866 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:59283/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:33813
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:02:42 dist-test-slave-trms krb5kdc[3584](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790334162, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 11:02:42.489601 4140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:02:42.490016 4140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:02:42.490173 4140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:02:42.501864 4140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:02:42.502017 4140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:02:42.502272 4140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:02:42.522291 4140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.159.212:59283/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:02:42.524328 4140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:02:42.526367 4140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:02:42.570103 4151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:02:42.571405 4150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:02:42.576311 4154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:02:42.577452 4140 server_base.cc:1034] running on GCE node
I20260925 11:02:42.578334 4140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:02:42.579769 4140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:02:42.581094 4140 hybrid_clock.cc:648] HybridClock initialized: now 1790334162580959 us; error 85 us; skew 500 ppm
I20260925 11:02:42.581596 4140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:02:42 dist-test-slave-trms krb5kdc[3584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334162, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:02:42.585974 4140 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:02:42.594684 4140 webserver.cc:460] Webserver started at http://127.24.159.193:37103/ using document root <none> and password file <none>
I20260925 11:02:42.595650 4140 fs_manager.cc:359] Metadata directory not provided
I20260925 11:02:42.595829 4140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:02:42.596148 4140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 11:02:42 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334162, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:02:42.613411 4140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd"
format_stamp: "Formatted at 2026-09-25 11:02:42 on dist-test-slave-trms"
server_key: "cbe08062a783ac25e123c33b281b1fcd"
server_key_iv: "00d2e1d40554ffbed610d24aad1d6e0a"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:42.614632 4140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd"
format_stamp: "Formatted at 2026-09-25 11:02:42 on dist-test-slave-trms"
server_key: "cbe08062a783ac25e123c33b281b1fcd"
server_key_iv: "00d2e1d40554ffbed610d24aad1d6e0a"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:42.621616 4140 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260925 11:02:42.654810 4160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:42.656468 4140 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.005s sys 0.005s
I20260925 11:02:42.656649 4140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd"
format_stamp: "Formatted at 2026-09-25 11:02:42 on dist-test-slave-trms"
server_key: "cbe08062a783ac25e123c33b281b1fcd"
server_key_iv: "00d2e1d40554ffbed610d24aad1d6e0a"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:42.656883 4140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:02:42.684197 4140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:02:42.689630 4140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:02:42.690074 4140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:02:42.691651 4140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:02:42.695443 4140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:02:42.695634 4140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:42.695849 4140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:02:42.695971 4140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:42.811559 4140 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:40883
I20260925 11:02:42.811574 4272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:40883 every 8 connection(s)
I20260925 11:02:42.813328 4140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:02:42.817816 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 4140
I20260925 11:02:42.818099 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:02:42.819068 25215 external_mini_cluster.cc:1428] Setting key e1caaa488da9860fcb09e911023135e7
Sep 25 11:02:42 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334162, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
Sep 25 11:02:42 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334162, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:02:42.878461 4275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:42.816121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:56399 (local address 127.24.159.254:33813)
0925 11:02:42.834748 (+ 18627us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:42.834755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:42.851649 (+ 16894us) server_negotiation.cc:407] Connection header received
0925 11:02:42.859009 (+ 7360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:42.859016 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:42.859132 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:42.859224 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:02:42.861707 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:42.862853 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:42.865368 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:42.865983 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:42.866079 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:42.870544 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:02:42.870551 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:02:42.870556 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:02:42.870597 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:02:42.874494 (+ 3897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:42.875563 (+ 1069us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:42.875569 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:42.875572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:42.875720 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:42.876277 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:42.876282 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:42.876285 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:42.876653 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:02:42.876775 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:42.877805 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0925 11:02:42.878039 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18482,"thread_start_us":18628,"threads_started":1}
I20260925 11:02:42.880931 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:42.816795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:33813 (local address 127.24.159.193:56399)
0925 11:02:42.851421 (+ 34626us) negotiation.cc:107] Waiting for socket to connect
0925 11:02:42.851490 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0925 11:02:42.858510 (+ 7020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:02:42.859858 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:02:42.859871 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:02:42.860541 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:02:42.861485 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:42.861515 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:42.863085 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:42.863091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:42.865179 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:42.865195 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:42.866446 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:42.866455 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:42.866746 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:42.867461 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:02:42.867499 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:02:42.870260 (+ 2761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:02:42.874757 (+ 4497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:02:42.874768 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:02:42.874786 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:02:42.875412 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:02:42.875877 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:02:42.875882 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:02:42.875885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:02:42.876143 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:02:42.876882 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:02:42.876891 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:02:42.877365 (+ 474us) client_negotiation.cc:769] Sending connection context
0925 11:02:42.878923 (+ 1558us) client_negotiation.cc:240] Negotiation successful
0925 11:02:42.879511 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34165,"thread_start_us":37294,"threads_started":1}
I20260925 11:02:42.883023 4273 heartbeater.cc:347] Connected to a master server at 127.24.159.254:33813
I20260925 11:02:42.883324 4273 heartbeater.cc:464] Registering TS with master...
I20260925 11:02:42.884121 4273 heartbeater.cc:511] Master 127.24.159.254:33813 requested a full tablet report, sending...
I20260925 11:02:42.886636 4060 ts_manager.cc:194] Registered new tserver with Master: 8c113643b0924ec2b0c8f0c80c7ce3bd (127.24.159.193:40883)
I20260925 11:02:42.888651 4060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:56399
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
W20260925 11:02:42.937147 25215 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.24.159.194: real 0.115s user 0.000s sys 0.018s
I20260925 11:02:42.938061 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:59283/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:33813
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:02:43 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334162, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
W20260925 11:02:43.500730 4279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:02:43.501112 4279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:02:43.501386 4279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:02:43.515002 4279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:02:43.515304 4279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:02:43.515623 4279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:02:43.532032 4279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.159.212:59283/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:02:43.533599 4279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:02:43.535310 4279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:02:43.558727 4294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:02:43.559065 4293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:02:43.560118 4296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:02:43.561400 4279 server_base.cc:1034] running on GCE node
I20260925 11:02:43.562075 4279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:02:43.563357 4279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:02:43.564683 4279 hybrid_clock.cc:648] HybridClock initialized: now 1790334163564578 us; error 54 us; skew 500 ppm
I20260925 11:02:43.565126 4279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:02:43 dist-test-slave-trms krb5kdc[3584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334163, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:02:43.569664 4279 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:02:43.572183 4279 webserver.cc:460] Webserver started at http://127.24.159.194:41575/ using document root <none> and password file <none>
I20260925 11:02:43.573156 4279 fs_manager.cc:359] Metadata directory not provided
I20260925 11:02:43.573350 4279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:02:43.573771 4279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 11:02:43 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334163, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:02:43.595049 4279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "2693e13a122d40788336791f516f4039"
format_stamp: "Formatted at 2026-09-25 11:02:43 on dist-test-slave-trms"
server_key: "ab71b18d9e4ee5a4223bc884fbaf1862"
server_key_iv: "53b19764bd9c1ec2463e36f934e52eb1"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:43.596583 4279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "2693e13a122d40788336791f516f4039"
format_stamp: "Formatted at 2026-09-25 11:02:43 on dist-test-slave-trms"
server_key: "ab71b18d9e4ee5a4223bc884fbaf1862"
server_key_iv: "53b19764bd9c1ec2463e36f934e52eb1"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:43.606567 4279 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.008s sys 0.000s
I20260925 11:02:43.632066 4303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:43.634488 4279 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.005s sys 0.004s
I20260925 11:02:43.634750 4279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "2693e13a122d40788336791f516f4039"
format_stamp: "Formatted at 2026-09-25 11:02:43 on dist-test-slave-trms"
server_key: "ab71b18d9e4ee5a4223bc884fbaf1862"
server_key_iv: "53b19764bd9c1ec2463e36f934e52eb1"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:43.635152 4279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:02:43.674878 4279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:02:43.682085 4279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:02:43.682744 4279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:02:43.684803 4279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:02:43.687956 4279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:02:43.688124 4279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:43.688342 4279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:02:43.688491 4279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:43.801141 4279 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:44349
I20260925 11:02:43.801234 4415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:44349 every 8 connection(s)
I20260925 11:02:43.802927 4279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:02:43.809716 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 4279
I20260925 11:02:43.809883 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:02:43.810557 25215 external_mini_cluster.cc:1428] Setting key 815b9ba7b464cf8e0811e2aed1853248
Sep 25 11:02:43 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334163, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
I20260925 11:02:43.849344 4419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:43.807276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:50855 (local address 127.24.159.254:33813)
0925 11:02:43.826701 (+ 19425us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:43.826708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:43.826727 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:02:43.826783 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:43.826787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:43.826960 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:43.827077 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:02:43.831348 (+ 4271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:43.832743 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:43.835543 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:43.836292 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:43.836408 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:43.841312 (+ 4904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:02:43.841319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:02:43.841323 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:02:43.841365 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:02:43.845187 (+ 3822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:43.846133 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:43.846139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:43.846142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:43.846333 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:43.846894 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:43.846899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:43.846902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:43.847260 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:02:43.847391 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:43.848461 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0925 11:02:43.848854 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19294,"thread_start_us":20512,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
I20260925 11:02:43.850368 4418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:43.806643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:33813 (local address 127.24.159.194:50855)
0925 11:02:43.823100 (+ 16457us) negotiation.cc:107] Waiting for socket to connect
0925 11:02:43.823160 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 11:02:43.825890 (+ 2730us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:02:43.829436 (+ 3546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:02:43.829448 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:02:43.830226 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:02:43.831137 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:43.831162 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:43.832974 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:43.832981 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:43.835362 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:43.835378 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:43.836801 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:43.836808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:43.837113 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:43.837957 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:02:43.837987 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:02:43.841030 (+ 3043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:02:43.845398 (+ 4368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:02:43.845407 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:02:43.845425 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:02:43.846004 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:02:43.846482 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:02:43.846486 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:02:43.846489 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:02:43.846770 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:02:43.847496 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:02:43.847504 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:02:43.847983 (+ 479us) client_negotiation.cc:769] Sending connection context
0925 11:02:43.848306 (+ 323us) client_negotiation.cc:240] Negotiation successful
0925 11:02:43.848877 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16101,"thread_start_us":15850,"threads_started":1}
I20260925 11:02:43.852288 4416 heartbeater.cc:347] Connected to a master server at 127.24.159.254:33813
I20260925 11:02:43.852604 4416 heartbeater.cc:464] Registering TS with master...
I20260925 11:02:43.853439 4416 heartbeater.cc:511] Master 127.24.159.254:33813 requested a full tablet report, sending...
I20260925 11:02:43.855154 4060 ts_manager.cc:194] Registered new tserver with Master: 2693e13a122d40788336791f516f4039 (127.24.159.194:44349)
I20260925 11:02:43.856282 4060 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:50855
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:02:43.885468 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:59283/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:33813
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:02:43.892623 4273 heartbeater.cc:503] Master 127.24.159.254:33813 was elected leader, sending a full tablet report...
W20260925 11:02:44.296187 4422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:02:44.296453 4422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:02:44.296550 4422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:02:44.303581 4422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:02:44.303788 4422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:02:44.303951 4422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:02:44.314008 4422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.159.212:59283/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:33813
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:02:44.315269 4422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:02:44.316506 4422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:02:44.329810 4428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:02:44.329856 4427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:02:44.329772 4430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:02:44.330580 4422 server_base.cc:1034] running on GCE node
I20260925 11:02:44.331138 4422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:02:44.332046 4422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:02:44.333250 4422 hybrid_clock.cc:648] HybridClock initialized: now 1790334164333200 us; error 45 us; skew 500 ppm
I20260925 11:02:44.333573 4422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:02:44 dist-test-slave-trms krb5kdc[3584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334164, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:02:44.336601 4422 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:02:44.337962 4422 webserver.cc:460] Webserver started at http://127.24.159.195:43083/ using document root <none> and password file <none>
I20260925 11:02:44.338692 4422 fs_manager.cc:359] Metadata directory not provided
I20260925 11:02:44.338806 4422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:02:44.339092 4422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 11:02:44 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334164, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:02:44.355355 4422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "f391bcbf9cfc433e88113df8cbe42166"
format_stamp: "Formatted at 2026-09-25 11:02:44 on dist-test-slave-trms"
server_key: "56dc6b51b37b62b0fa1814a99bc176bc"
server_key_iv: "16fdfcc9303aa12bb8ccad3e4d571f9c"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:44.356417 4422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "f391bcbf9cfc433e88113df8cbe42166"
format_stamp: "Formatted at 2026-09-25 11:02:44 on dist-test-slave-trms"
server_key: "56dc6b51b37b62b0fa1814a99bc176bc"
server_key_iv: "16fdfcc9303aa12bb8ccad3e4d571f9c"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:44.363957 4422 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260925 11:02:44.379248 4437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:44.380853 4422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:02:44.381027 4422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "f391bcbf9cfc433e88113df8cbe42166"
format_stamp: "Formatted at 2026-09-25 11:02:44 on dist-test-slave-trms"
server_key: "56dc6b51b37b62b0fa1814a99bc176bc"
server_key_iv: "16fdfcc9303aa12bb8ccad3e4d571f9c"
server_key_version: "kuduclusterkey@0"
I20260925 11:02:44.381240 4422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:02:44.415928 4422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:02:44.419732 4422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:02:44.419998 4422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:02:44.420965 4422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:02:44.422856 4422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:02:44.422935 4422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:44.423053 4422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:02:44.423102 4422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:02:44.456369 4422 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:39435
I20260925 11:02:44.456405 4549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:39435 every 8 connection(s)
I20260925 11:02:44.457619 4422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:02:44.464159 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 4422
I20260925 11:02:44.464334 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:02:44.464847 25215 external_mini_cluster.cc:1428] Setting key 7cf6417b9951489ad0323e83b1eb5c96
Sep 25 11:02:44 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334164, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:02:44.476881 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.459974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:60113 (local address 127.24.159.254:33813)
0925 11:02:44.460517 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:44.460522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:44.461531 (+ 1009us) server_negotiation.cc:407] Connection header received
0925 11:02:44.463293 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:44.463298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:44.463374 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:44.463464 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:02:44.465216 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.466048 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.468538 (+ 2490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.468968 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.469072 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.472088 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:02:44.472093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:02:44.472096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:02:44.472125 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:02:44.474500 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:44.475115 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:44.475119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:44.475121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:44.475231 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:44.475565 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:44.475568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:44.475570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:44.475822 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:02:44.475925 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:44.476481 (+ 556us) server_negotiation.cc:299] Negotiation successful
0925 11:02:44.476631 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":299,"threads_started":1}
I20260925 11:02:44.477850 4553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.460615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:33813 (local address 127.24.159.195:60113)
0925 11:02:44.461384 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0925 11:02:44.461421 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 11:02:44.463046 (+ 1625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:02:44.463694 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:02:44.463702 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:02:44.464242 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:02:44.465037 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.465065 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.466237 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.466242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.468405 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.468417 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.469147 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.469152 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.469347 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.469941 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:02:44.469965 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:02:44.471887 (+ 1922us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:02:44.474617 (+ 2730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:02:44.474624 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:02:44.474636 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:02:44.475029 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:02:44.475303 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:02:44.475306 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:02:44.475308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:02:44.475484 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:02:44.475908 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:02:44.475914 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:02:44.476213 (+ 299us) client_negotiation.cc:769] Sending connection context
0925 11:02:44.476415 (+ 202us) client_negotiation.cc:240] Negotiation successful
0925 11:02:44.476766 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":326,"threads_started":1}
I20260925 11:02:44.479226 4550 heartbeater.cc:347] Connected to a master server at 127.24.159.254:33813
I20260925 11:02:44.479533 4550 heartbeater.cc:464] Registering TS with master...
I20260925 11:02:44.480249 4550 heartbeater.cc:511] Master 127.24.159.254:33813 requested a full tablet report, sending...
I20260925 11:02:44.481580 4059 ts_manager.cc:194] Registered new tserver with Master: f391bcbf9cfc433e88113df8cbe42166 (127.24.159.195:39435)
I20260925 11:02:44.482467 4059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:60113
I20260925 11:02:44.492033 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:02:44.508051 4552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.496394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36170 (local address 127.24.159.254:33813)
0925 11:02:44.496550 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:44.496554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:44.497488 (+ 934us) server_negotiation.cc:407] Connection header received
0925 11:02:44.497618 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:44.497621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:44.497698 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:44.497776 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:02:44.499089 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.499771 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.501011 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.501402 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.501493 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.503709 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:02:44.503714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:02:44.503717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:02:44.503744 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:02:44.505894 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:44.506524 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:44.506528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:44.506530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:44.506646 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:44.506930 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:44.506933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:44.506935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:44.507190 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:02:44.507356 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:44.507583 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 11:02:44.507813 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:02:44.512072 4059 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:02:44.519663 4059 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:02:44.551391 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.532304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.24.159.194:44349)
0925 11:02:44.538672 (+ 6368us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:44.538683 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:44.542030 (+ 3347us) server_negotiation.cc:407] Connection header received
0925 11:02:44.542286 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:44.542294 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:44.542541 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:44.542737 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:02:44.543837 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.545402 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.548883 (+ 3481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.549806 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.549959 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.550157 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:44.550859 (+ 702us) server_negotiation.cc:299] Negotiation successful
0925 11:02:44.551047 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6186,"thread_start_us":6058,"threads_started":1}
I20260925 11:02:44.551391 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.532790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.24.159.195:39435)
0925 11:02:44.541414 (+ 8624us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:44.541422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:44.541456 (+ 34us) server_negotiation.cc:407] Connection header received
0925 11:02:44.541561 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:44.541570 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:44.541920 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:44.543142 (+ 1222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:02:44.543962 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.545462 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.548394 (+ 2932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.549907 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.550403 (+ 496us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.550543 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:44.550929 (+ 386us) server_negotiation.cc:299] Negotiation successful
0925 11:02:44.551077 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8463,"thread_start_us":8303,"threads_started":1}
I20260925 11:02:44.551395 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.532136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:44349 (local address 127.0.0.1:35754)
0925 11:02:44.541790 (+ 9654us) negotiation.cc:107] Waiting for socket to connect
0925 11:02:44.541826 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0925 11:02:44.541938 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:02:44.542971 (+ 1033us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:02:44.542975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:02:44.543000 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:02:44.543491 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.543505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.545583 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.545589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.548743 (+ 3154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.548752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.550376 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.550383 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.550577 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.550645 (+ 68us) client_negotiation.cc:769] Sending connection context
0925 11:02:44.550785 (+ 140us) client_negotiation.cc:240] Negotiation successful
0925 11:02:44.550987 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9533,"mutex_wait_us":74,"thread_start_us":9293,"threads_started":1}
I20260925 11:02:44.551788 4563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.532136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:39435 (local address 127.0.0.1:54708)
0925 11:02:44.532928 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0925 11:02:44.532973 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0925 11:02:44.533181 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:02:44.542362 (+ 9181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:02:44.542370 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:02:44.542400 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:02:44.542873 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.542885 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.545594 (+ 2709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.545599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.548241 (+ 2642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.548255 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.550624 (+ 2369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.550632 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.550788 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.550816 (+ 28us) client_negotiation.cc:769] Sending connection context
0925 11:02:44.550888 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 11:02:44.551345 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":313,"thread_start_us":883,"threads_started":1}
I20260925 11:02:44.556026 4568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.542276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55300 (local address 127.24.159.193:40883)
0925 11:02:44.547079 (+ 4803us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:44.547086 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:44.547108 (+ 22us) server_negotiation.cc:407] Connection header received
0925 11:02:44.547182 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:44.547187 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:44.547371 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:44.547551 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:02:44.548203 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.549775 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.554045 (+ 4270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.555022 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.555201 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.555372 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:44.555550 (+ 178us) server_negotiation.cc:299] Negotiation successful
0925 11:02:44.555704 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4587,"spinlock_wait_cycles":27904,"thread_start_us":4555,"threads_started":1}
I20260925 11:02:44.555686 4351 tablet_service.cc:1467] Processing CreateTablet for tablet df038adea8284d439063816be370472a (DEFAULT_TABLE table=test-table [id=b3913182cf72431eb1384bb7e8b1b8fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:02:44.555689 4485 tablet_service.cc:1467] Processing CreateTablet for tablet df038adea8284d439063816be370472a (DEFAULT_TABLE table=test-table [id=b3913182cf72431eb1384bb7e8b1b8fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:02:44.556612 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.542125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:40883 (local address 127.0.0.1:55300)
0925 11:02:44.546731 (+ 4606us) negotiation.cc:107] Waiting for socket to connect
0925 11:02:44.546752 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:02:44.546864 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:02:44.547545 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:02:44.547550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:02:44.547575 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:02:44.548051 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.548064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.550243 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.550249 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.553866 (+ 3617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.553878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.555221 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.555228 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.555390 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.555423 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 11:02:44.555500 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 11:02:44.555629 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4442,"mutex_wait_us":3,"thread_start_us":4295,"threads_started":1}
I20260925 11:02:44.557225 4351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df038adea8284d439063816be370472a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:02:44.557226 4485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df038adea8284d439063816be370472a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:02:44.559742 4208 tablet_service.cc:1467] Processing CreateTablet for tablet df038adea8284d439063816be370472a (DEFAULT_TABLE table=test-table [id=b3913182cf72431eb1384bb7e8b1b8fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:02:44.560901 4208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df038adea8284d439063816be370472a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:02:44.571282 4569 tablet_bootstrap.cc:492] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: Bootstrap starting.
I20260925 11:02:44.575310 4569 tablet_bootstrap.cc:654] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: Neither blocks nor log segments found. Creating new log.
I20260925 11:02:44.576310 4569 log.cc:824] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: Log is configured to *not* fsync() on all Append() calls
I20260925 11:02:44.578480 4571 tablet_bootstrap.cc:492] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: Bootstrap starting.
I20260925 11:02:44.579095 4569 tablet_bootstrap.cc:492] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: No bootstrap required, opened a new log
I20260925 11:02:44.579324 4569 ts_tablet_manager.cc:1397] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260925 11:02:44.581236 4571 tablet_bootstrap.cc:654] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: Neither blocks nor log segments found. Creating new log.
I20260925 11:02:44.581547 4570 tablet_bootstrap.cc:492] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: Bootstrap starting.
I20260925 11:02:44.582294 4571 log.cc:824] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: Log is configured to *not* fsync() on all Append() calls
I20260925 11:02:44.585198 4570 tablet_bootstrap.cc:654] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: Neither blocks nor log segments found. Creating new log.
I20260925 11:02:44.585903 4571 tablet_bootstrap.cc:492] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: No bootstrap required, opened a new log
I20260925 11:02:44.586139 4571 ts_tablet_manager.cc:1397] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260925 11:02:44.586419 4570 log.cc:824] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: Log is configured to *not* fsync() on all Append() calls
I20260925 11:02:44.589165 4569 raft_consensus.cc:348] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } }
I20260925 11:02:44.589609 4569 raft_consensus.cc:374] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:02:44.589751 4569 raft_consensus.cc:729] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c113643b0924ec2b0c8f0c80c7ce3bd, State: Initialized, Role: FOLLOWER
I20260925 11:02:44.590050 4570 tablet_bootstrap.cc:492] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: No bootstrap required, opened a new log
I20260925 11:02:44.590391 4570 ts_tablet_manager.cc:1397] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.003s
I20260925 11:02:44.590391 4569 consensus_queue.cc:260] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } }
I20260925 11:02:44.592038 4569 ts_tablet_manager.cc:1428] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260925 11:02:44.594040 4571 raft_consensus.cc:348] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } }
I20260925 11:02:44.594424 4571 raft_consensus.cc:374] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:02:44.594527 4571 raft_consensus.cc:729] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f391bcbf9cfc433e88113df8cbe42166, State: Initialized, Role: FOLLOWER
I20260925 11:02:44.595094 4571 consensus_queue.cc:260] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } }
I20260925 11:02:44.597168 4550 heartbeater.cc:503] Master 127.24.159.254:33813 was elected leader, sending a full tablet report...
I20260925 11:02:44.597887 4571 ts_tablet_manager.cc:1428] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260925 11:02:44.600445 4570 raft_consensus.cc:348] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } }
I20260925 11:02:44.600780 4570 raft_consensus.cc:374] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:02:44.600864 4570 raft_consensus.cc:729] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2693e13a122d40788336791f516f4039, State: Initialized, Role: FOLLOWER
I20260925 11:02:44.601372 4570 consensus_queue.cc:260] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } }
I20260925 11:02:44.602339 4416 heartbeater.cc:503] Master 127.24.159.254:33813 was elected leader, sending a full tablet report...
I20260925 11:02:44.602921 4570 ts_tablet_manager.cc:1428] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
W20260925 11:02:44.610292 4274 tablet.cc:2369] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:02:44.709115 4551 tablet.cc:2369] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:02:44.781145 4575 raft_consensus.cc:482] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:02:44.781457 4575 raft_consensus.cc:504] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } }
I20260925 11:02:44.783149 4575 leader_election.cc:288] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2693e13a122d40788336791f516f4039 (127.24.159.194:44349), f391bcbf9cfc433e88113df8cbe42166 (127.24.159.195:39435)
I20260925 11:02:44.789912 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.783648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:55017 (local address 127.24.159.195:39435)
0925 11:02:44.783844 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:44.783850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:44.784746 (+ 896us) server_negotiation.cc:407] Connection header received
0925 11:02:44.784939 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:44.784943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:44.785006 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:44.785076 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:02:44.785672 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.786409 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.788257 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.789049 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.789196 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.789274 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:44.789552 (+ 278us) server_negotiation.cc:299] Negotiation successful
0925 11:02:44.789677 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 11:02:44.789970 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.783500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:33369 (local address 127.24.159.194:44349)
0925 11:02:44.783712 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:44.783719 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:44.784914 (+ 1195us) server_negotiation.cc:407] Connection header received
0925 11:02:44.785079 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:44.785083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:44.785139 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:44.785211 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:02:44.785884 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.786650 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.788368 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.789121 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.789256 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.789328 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:44.789615 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 11:02:44.789734 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:02:44.790123 4579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.783676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:44349 (local address 127.24.159.193:33369)
0925 11:02:44.784814 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0925 11:02:44.784830 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:02:44.784955 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:02:44.785238 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:02:44.785243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:02:44.785355 (+ 112us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:02:44.785734 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.785744 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.786779 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.786787 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.788221 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.788232 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.789326 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.789336 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.789467 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.789489 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 11:02:44.789571 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 11:02:44.789695 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1014,"mutex_wait_us":36,"spinlock_wait_cycles":9088,"thread_start_us":805,"threads_started":1}
I20260925 11:02:44.790176 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.783392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:39435 (local address 127.24.159.193:55017)
0925 11:02:44.784639 (+ 1247us) negotiation.cc:107] Waiting for socket to connect
0925 11:02:44.784658 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:02:44.784802 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:02:44.785107 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:02:44.785111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:02:44.785135 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:02:44.785536 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.785546 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.786535 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.786540 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.788100 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:02:44.788113 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.789250 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.789256 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:02:44.789406 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.789430 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 11:02:44.789508 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 11:02:44.789644 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1127,"mutex_wait_us":41,"thread_start_us":938,"threads_started":1}
I20260925 11:02:44.791149 4505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df038adea8284d439063816be370472a" candidate_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f391bcbf9cfc433e88113df8cbe42166" is_pre_election: true
I20260925 11:02:44.791150 4371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df038adea8284d439063816be370472a" candidate_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2693e13a122d40788336791f516f4039" is_pre_election: true
I20260925 11:02:44.791692 4371 raft_consensus.cc:2454] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c113643b0924ec2b0c8f0c80c7ce3bd in term 0.
I20260925 11:02:44.791752 4505 raft_consensus.cc:2454] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c113643b0924ec2b0c8f0c80c7ce3bd in term 0.
I20260925 11:02:44.792426 4163 leader_election.cc:302] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2693e13a122d40788336791f516f4039, 8c113643b0924ec2b0c8f0c80c7ce3bd; no voters:
I20260925 11:02:44.792842 4575 raft_consensus.cc:2781] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:02:44.793116 4575 raft_consensus.cc:482] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:02:44.793212 4575 raft_consensus.cc:3037] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:02:44.795872 4575 raft_consensus.cc:504] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } }
I20260925 11:02:44.796454 4575 leader_election.cc:288] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [CANDIDATE]: Term 1 election: Requested vote from peers 2693e13a122d40788336791f516f4039 (127.24.159.194:44349), f391bcbf9cfc433e88113df8cbe42166 (127.24.159.195:39435)
I20260925 11:02:44.796886 4371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df038adea8284d439063816be370472a" candidate_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2693e13a122d40788336791f516f4039"
I20260925 11:02:44.796887 4505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df038adea8284d439063816be370472a" candidate_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f391bcbf9cfc433e88113df8cbe42166"
I20260925 11:02:44.797058 4505 raft_consensus.cc:3037] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:02:44.797065 4371 raft_consensus.cc:3037] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:02:44.799229 4505 raft_consensus.cc:2454] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c113643b0924ec2b0c8f0c80c7ce3bd in term 1.
I20260925 11:02:44.799311 4371 raft_consensus.cc:2454] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c113643b0924ec2b0c8f0c80c7ce3bd in term 1.
I20260925 11:02:44.799646 4161 leader_election.cc:302] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c113643b0924ec2b0c8f0c80c7ce3bd, f391bcbf9cfc433e88113df8cbe42166; no voters:
I20260925 11:02:44.799965 4575 raft_consensus.cc:2781] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:02:44.800433 4575 raft_consensus.cc:686] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 1 LEADER]: Becoming Leader. State: Replica: 8c113643b0924ec2b0c8f0c80c7ce3bd, State: Running, Role: LEADER
I20260925 11:02:44.800982 4575 consensus_queue.cc:237] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } }
I20260925 11:02:44.808257 4059 catalog_manager.cc:5495] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c113643b0924ec2b0c8f0c80c7ce3bd (127.24.159.193). New cstate: current_term: 1 leader_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c113643b0924ec2b0c8f0c80c7ce3bd" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 40883 } health_report { overall_health: HEALTHY } } }
W20260925 11:02:44.828835 4417 tablet.cc:2369] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:02:44.839798 4568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.834578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55304 (local address 127.24.159.193:40883)
0925 11:02:44.834923 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:44.834930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:44.834949 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:02:44.835049 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:44.835053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:44.835128 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:44.835260 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:02:44.835642 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.836368 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.837766 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.838149 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.838302 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.838757 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:02:44.839255 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:02:44.839385 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:44.839478 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 11:02:44.839564 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 11:02:44.847613 4505 raft_consensus.cc:1261] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [term 1 FOLLOWER]: Refusing update from remote peer 8c113643b0924ec2b0c8f0c80c7ce3bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:02:44.847613 4371 raft_consensus.cc:1261] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [term 1 FOLLOWER]: Refusing update from remote peer 8c113643b0924ec2b0c8f0c80c7ce3bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:02:44.848519 4575 consensus_queue.cc:1035] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2693e13a122d40788336791f516f4039" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 44349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:02:44.848814 4580 consensus_queue.cc:1035] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f391bcbf9cfc433e88113df8cbe42166" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:02:44.858906 4583 mvcc.cc:205] Tried to move back new op lower bound from 7333208739206639616 to 7333208739028606976. Current Snapshot: MvccSnapshot[applied={T|T < 7333208739206639616}]
I20260925 11:02:44.863896 4585 mvcc.cc:205] Tried to move back new op lower bound from 7333208739206639616 to 7333208739028606976. Current Snapshot: MvccSnapshot[applied={T|T < 7333208739206639616}]
I20260925 11:02:44.874714 4060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:02:44.874986 4060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:02:44.878720 4060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9aeddca43fc849c490342bf985f75f1b: Sending DeleteTablet for 3 replicas of tablet df038adea8284d439063816be370472a
I20260925 11:02:44.879853 4485 tablet_service.cc:1514] Processing DeleteTablet for tablet df038adea8284d439063816be370472a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:02:44 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:54708
I20260925 11:02:44.879868 4351 tablet_service.cc:1514] Processing DeleteTablet for tablet df038adea8284d439063816be370472a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:02:44 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35754
I20260925 11:02:44.880223 4208 tablet_service.cc:1514] Processing DeleteTablet for tablet df038adea8284d439063816be370472a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:02:44 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:55300
I20260925 11:02:44.882339 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 4140
I20260925 11:02:44.895809 4593 tablet_replica.cc:330] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: stopping tablet replica
I20260925 11:02:44.896312 4593 raft_consensus.cc:2229] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:02:44.896837 4593 raft_consensus.cc:2258] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:02:44.900842 4593 ts_tablet_manager.cc:1905] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:02:44.900908 4594 tablet_replica.cc:330] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: stopping tablet replica
I20260925 11:02:44.901332 4594 raft_consensus.cc:2229] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:02:44.902050 4594 raft_consensus.cc:2258] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:02:44.902827 4592 tablet_replica.cc:330] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: stopping tablet replica
I20260925 11:02:44.903270 4592 raft_consensus.cc:2229] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:02:44.903777 4592 raft_consensus.cc:2258] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:02:44.905787 4594 ts_tablet_manager.cc:1905] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:02:44.907543 4592 ts_tablet_manager.cc:1905] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 11:02:44 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:02:44.913087 4594 ts_tablet_manager.cc:1918] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:02:44.913304 4594 log.cc:1196] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/df038adea8284d439063816be370472a
I20260925 11:02:44.913460 4593 ts_tablet_manager.cc:1918] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:02:44.913699 4593 log.cc:1196] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/df038adea8284d439063816be370472a
I20260925 11:02:44.913904 4594 ts_tablet_manager.cc:1939] T df038adea8284d439063816be370472a P 8c113643b0924ec2b0c8f0c80c7ce3bd: Deleting consensus metadata
I20260925 11:02:44.915830 4046 catalog_manager.cc:4841] TS 8c113643b0924ec2b0c8f0c80c7ce3bd (127.24.159.193:40883): tablet df038adea8284d439063816be370472a (table test-table [id=b3913182cf72431eb1384bb7e8b1b8fe]) successfully deleted
I20260925 11:02:44.916054 4593 ts_tablet_manager.cc:1939] T df038adea8284d439063816be370472a P 2693e13a122d40788336791f516f4039: Deleting consensus metadata
I20260925 11:02:44.917752 4046 catalog_manager.cc:4841] TS 2693e13a122d40788336791f516f4039 (127.24.159.194:44349): tablet df038adea8284d439063816be370472a (table test-table [id=b3913182cf72431eb1384bb7e8b1b8fe]) successfully deleted
I20260925 11:02:44.919248 4592 ts_tablet_manager.cc:1918] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:02:44.919436 4592 log.cc:1196] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/df038adea8284d439063816be370472a
I20260925 11:02:44.919744 4568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:44.883763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55308 (local address 127.24.159.193:40883)
0925 11:02:44.883946 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:44.883952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:44.900225 (+ 16273us) server_negotiation.cc:407] Connection header received
0925 11:02:44.900430 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:44.900435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:44.900565 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:44.900679 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:02:44.902431 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.903549 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.905882 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:44.906557 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:44.906675 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:44.911814 (+ 5139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:02:44.911828 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:02:44.911845 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:02:44.911892 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:02:44.916133 (+ 4241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:44.917039 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:44.917048 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:44.917052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:44.917219 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:44.917678 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:44.917683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:44.917686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:44.918172 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:02:44.918539 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:44.919167 (+ 628us) server_negotiation.cc:299] Negotiation successful
0925 11:02:44.919394 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:02:44.919870 4592 ts_tablet_manager.cc:1939] T df038adea8284d439063816be370472a P f391bcbf9cfc433e88113df8cbe42166: Deleting consensus metadata
I20260925 11:02:44.921221 4268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:02:44.921512 4044 catalog_manager.cc:4841] TS f391bcbf9cfc433e88113df8cbe42166 (127.24.159.195:39435): tablet df038adea8284d439063816be370472a (table test-table [id=b3913182cf72431eb1384bb7e8b1b8fe]) successfully deleted
I20260925 11:02:45.018431 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 4140
I20260925 11:02:45.033514 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 4279
Sep 25 11:02:45 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:02:45.047315 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:45.034265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.24.159.194:44349)
0925 11:02:45.034408 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:45.034413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:45.034466 (+ 53us) server_negotiation.cc:407] Connection header received
0925 11:02:45.034615 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:45.034619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:45.034695 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:45.034782 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:02:45.035957 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:45.036650 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:45.037864 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:45.038306 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:45.038409 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:45.042297 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:02:45.042310 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:02:45.042325 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:02:45.042366 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:02:45.045026 (+ 2660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:45.045655 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:45.045661 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:45.045664 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:45.045784 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:45.046063 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:45.046066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:45.046068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:45.046437 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:02:45.046649 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:45.046901 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 11:02:45.047060 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 11:02:45.047922 4411 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:02:45.134595 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 4279
I20260925 11:02:45.147333 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 4422
Sep 25 11:02:45 dist-test-slave-trms krb5kdc[3584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:02:45.161062 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:02:45.148069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54722 (local address 127.24.159.195:39435)
0925 11:02:45.148246 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:02:45.148251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:02:45.148269 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:02:45.148419 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:02:45.148423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:02:45.148485 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:02:45.148567 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:02:45.149744 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:45.150518 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:45.151877 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:02:45.152260 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:02:45.152357 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:02:45.156291 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:02:45.156307 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:02:45.156324 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:02:45.156365 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:02:45.158845 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:45.159425 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:45.159432 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:45.159435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:45.159546 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:02:45.159805 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:02:45.159808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:02:45.159809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:02:45.160161 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:02:45.160373 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0925 11:02:45.160589 (+ 216us) server_negotiation.cc:299] Negotiation successful
0925 11:02:45.160737 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:02:45.161633 4545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:02:45.247933 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 4422
I20260925 11:02:45.260555 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 4028
I20260925 11:02:45.261615 4090 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:02:45.371330 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 4028
I20260925 11:02:45.388439 25215 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 11:02:45.723402 25215 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 11:02:45.723618 25215 mini_ranger.cc:66] Stopping Ranger...
I20260925 11:02:46.058092 25215 mini_ranger.cc:68] Stopped Ranger
2026-09-25 11:02:46.058 UTC [3614] LOG: received smart shutdown request
2026-09-25 11:02:46.061 UTC [3614] LOG: background worker "logical replication launcher" (PID 3623) exited with exit code 1
2026-09-25 11:02:46.061 UTC [3618] LOG: shutting down
2026-09-25 11:02:46.218 UTC [3614] LOG: database system is shut down
2026-09-25T11:02:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46552 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:02:46 dist-test-slave-trms krb5kdc[4608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:02:46 dist-test-slave-trms krb5kdc[4608](info): set up 2 sockets
Sep 25 11:02:46 dist-test-slave-trms krb5kdc[4608](info): commencing operation
krb5kdc: starting...
W20260925 11:02:47.338474 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:02:47 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:02:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:02:47Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.159.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.159.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/rangeradmin_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/rangeradmin_127.24.159.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.159.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.159.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/rangerlookup_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/rangerlookup_127.24.159.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.159.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.159.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/HTTP_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/HTTP_127.24.159.212@KRBTEST.COM.keytab.
I20260925 11:02:47.626778 25215 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskw_hhNs/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/postgres -l logfile start
2026-09-25 11:02:49.861 UTC [4638] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-25 11:02:49.861 UTC [4638] LOG: listening on IPv4 address "127.24.159.212", port 36847
2026-09-25 11:02:49.864 UTC [4638] LOG: listening on Unix socket "/tmp/.s.PGSQL.36847"
2026-09-25 11:02:49.883 UTC [4641] LOG: database system was shut down at 2026-09-25 11:02:48 UTC
2026-09-25 11:02:49.887 UTC [4638] LOG: database system is ready to accept connections
I20260925 11:02:50.770252 25215 mini_postgres.cc:96] Postgres bound to 36847
2026-09-25 11:02:50.786 UTC [4649] FATAL: database "slave" does not exist
127.24.159.212:36847 - accepting connections
I20260925 11:02:50.786598 25215 mini_ranger.cc:161] Starting Ranger...
I20260925 11:02:50.810377 25215 mini_ranger.cc:84] Created miniranger Postgres user
I20260925 11:02:51.217707 25215 mini_ranger.cc:87] Created ranger Postgres database
I20260925 11:02:51.217821 25215 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-admin
2026-09-25 11:02:51,473 [I] DB FLAVOR :POSTGRES
2026-09-25 11:02:51,474 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 11:02:51,474 [I] Checking connection..
2026-09-25 11:02:51,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-25 11:02:51,726 [I] Checking connection passed.
2026-09-25 11:02:51,726 [I] --------- Verifying version history table ---------
2026-09-25 11:02:51,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 11:02:51,970 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 11:02:51,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 11:02:52,221 [I] Table x_db_version_h does not exist in database ranger
2026-09-25 11:02:52,221 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-25 11:02:52,221 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-25 11:02:52.470 UTC [4713] WARNING: there is no transaction in progress
2026-09-25 11:02:52,475 [I] create_dbversion_catalog.sql file imported successfully
2026-09-25 11:02:52,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-25 11:02:52,704 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-25 11:02:52,705 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 11:02:52,705 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-25 11:02:52,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-25 11:02:53,158 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-trms.c.gcp-upstream.internal','N') ;"
2026-09-25 11:02:53,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 11:02:53,622 [I] Table x_portal_user does not exist in database ranger
2026-09-25 11:02:53,622 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 11:02:53,842 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-25 11:02:53,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 11:02:54,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 11:02:54,325 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-25 11:02:54,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-25 11:02:55.219 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.234 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.248 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.262 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.276 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.293 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.344 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.350 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.363 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.372 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.381 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.395 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.404 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.409 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.415 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55.420 UTC [4821] WARNING: there is no transaction in progress
2026-09-25 11:02:55,982 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-25 11:02:55,982 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-25 11:02:56,212 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-25 11:02:56,212 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-25 11:02:56,463 [I] Table x_portal_user already exists in database 'ranger'
2026-09-25 11:02:56,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-25 11:02:56,684 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-25 11:02:56,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-25 11:02:56,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-25 11:02:57,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-trms.c.gcp-upstream.internal';"
2026-09-25 11:02:57,376 [I] CORE_DB_SCHEMA import status has been updated
2026-09-25 11:02:57,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-25 11:02:57,611 [I] DB_PATCHES have already been applied
I20260925 11:02:57.617015 25215 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-admin:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskw_hhNs/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260925 11:02:57.617100 25215 mini_ranger.cc:193] Using host: 127.24.159.212
I20260925 11:02:57.630726 25215 mini_ranger.cc:227] Ranger admin URL: http://127.24.159.212:32875
Sep 25, 2026 11:02:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 11:02:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 11:02:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-admin/ews/webapp
Sep 25, 2026 11:02:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 25, 2026 11:02:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 25, 2026 11:02:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-admin/ews/webapp].
Sep 25, 2026 11:02:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260925 11:02:58.655402 25215 mini_ranger.cc:160] Time spent starting Ranger: real 7.869s user 0.000s sys 0.037s
Sep 25, 2026 11:03:18 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 25, 2026 11:03:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 25, 2026 11:03:18 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 25, 2026 11:03:18 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 25, 2026 11:03:19 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260925 11:03:20.008035 25215 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.159.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.159.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/rangerkms_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.159.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/rangerkms_127.24.159.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.159.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.159.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.159.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/HTTP_127.24.159.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.159.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/HTTP_127.24.159.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 25 11:03:20 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334200, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260925 11:03:20.156322 25215 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260925 11:03:20.175179 25215 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260925 11:03:20.929190 25215 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260925 11:03:20.929286 25215 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-kms
I20260925 11:03:20.929327 25215 mini_ranger_kms.cc:227] Using postgres at 127.24.159.212:36847
2026-09-25 11:03:21,261 [I] DB FLAVOR :POSTGRES
2026-09-25 11:03:21,261 [I] --------- Verifying Ranger DB connection ---------
2026-09-25 11:03:21,261 [I] Checking connection
2026-09-25 11:03:21,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-25 11:03:21,508 [I] connection success
2026-09-25 11:03:21,508 [I] --------- Verifying Ranger DB tables ---------
2026-09-25 11:03:21,508 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-25 11:03:21,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-25 11:03:21,759 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-25 11:03:21,760 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-25 11:03:21,760 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-25 11:03:21,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.159.212:36847/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-25 11:03:22,014 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260925 11:03:22.456374 25215 mini_ranger_kms.cc:310] Created kms service
I20260925 11:03:22.645205 25215 mini_ranger_kms.cc:326] Created kudu user
I20260925 11:03:22.690462 25215 mini_ranger_kms.cc:343] Created rangerkms user
Sep 25 11:03:22 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334200, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:23.062383 25215 mini_ranger_kms.cc:384] Added ranger policy
I20260925 11:03:23.062685 25215 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-kms:/tmp/dist-test-taskw_hhNs/build/asan/bin/postgresql.jar:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskw_hhNs/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260925 11:03:23.062742 25215 mini_ranger_kms.cc:240] Using host: 127.24.159.212
I20260925 11:03:23.077492 25215 mini_ranger_kms.cc:276] Ranger KMS PID: 5001
I20260925 11:03:23.077615 25215 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.159.212:47461
Sep 25, 2026 11:03:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 25, 2026 11:03:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 25, 2026 11:03:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 25, 2026 11:03:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 25, 2026 11:03:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-kms/ews/webapp].
Sep 25, 2026 11:03:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260925 11:03:25.081171 25215 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.925s user 0.074s sys 0.273s
I20260925 11:03:25.081422 25215 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260925 11:03:25.081614 25215 mini_ranger_kms.cc:401] 127.24.159.212:47461/kms/v1/keys
Sep 25 11:03:25 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790334205, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.159.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 25 11:03:25 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334205, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.159.212@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
Sep 25 11:03:26 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334205, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.159.212@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
Sep 25 11:03:28 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:03:28.307612 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43055
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:43055
--ranger_config_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:47461/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:28.626255 5052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:28.626543 5052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:28.626668 5052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:28.633843 5052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:03:28.633949 5052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:28.634007 5052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:28.634090 5052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:03:28.634147 5052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:03:28.644179 5052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.159.212:47461/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:43055
--ranger_config_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:28.645426 5052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:28.646764 5052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:28.659804 5057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:28.659775 5058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:28.660650 5060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:28.660948 5052 server_base.cc:1034] running on GCE node
I20260925 11:03:28.661850 5052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:28.663580 5052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:28.664825 5052 hybrid_clock.cc:648] HybridClock initialized: now 1790334208664775 us; error 29 us; skew 500 ppm
I20260925 11:03:28.665225 5052 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:28 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334208, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:28.669029 5052 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:03:28.670727 5052 webserver.cc:460] Webserver started at http://127.24.159.254:44667/ using document root <none> and password file <none>
I20260925 11:03:28.671526 5052 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:28.671660 5052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:28.672029 5052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 11:03:28 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334208, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:28.722304 5052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "fb48ab0868d5424f9ad7626c304e89e6"
format_stamp: "Formatted at 2026-09-25 11:03:28 on dist-test-slave-trms"
server_key: "b95ae634397bc82101672e6aac7b0eb5"
server_key_iv: "35a197dc5368308ff0b84508551c7bce"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:28.723374 5052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "fb48ab0868d5424f9ad7626c304e89e6"
format_stamp: "Formatted at 2026-09-25 11:03:28 on dist-test-slave-trms"
server_key: "b95ae634397bc82101672e6aac7b0eb5"
server_key_iv: "35a197dc5368308ff0b84508551c7bce"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:28.730265 5052 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260925 11:03:28.749962 5067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:28.751739 5052 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 11:03:28.751940 5052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "fb48ab0868d5424f9ad7626c304e89e6"
format_stamp: "Formatted at 2026-09-25 11:03:28 on dist-test-slave-trms"
server_key: "b95ae634397bc82101672e6aac7b0eb5"
server_key_iv: "35a197dc5368308ff0b84508551c7bce"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:28.752184 5052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:28.775235 5052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:28.779551 5052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:28.779832 5052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:28.798048 5052 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:43055
I20260925 11:03:28.798092 5118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:43055 every 8 connection(s)
I20260925 11:03:28.799399 5052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:03:28.800235 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 5052
I20260925 11:03:28.800480 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:03:28.801318 25215 external_mini_cluster.cc:1428] Setting key 9370cc1e1351e20b2b4d04408651249f
I20260925 11:03:28.810505 5119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 11:03:28 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:28.828037 5119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Bootstrap starting.
I20260925 11:03:28.832020 5119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:28.832389 5122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:28.803400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46244 (local address 127.24.159.254:43055)
0925 11:03:28.814615 (+ 11215us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:28.814628 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:28.814671 (+ 43us) server_negotiation.cc:407] Connection header received
0925 11:03:28.816316 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:28.816339 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:28.816755 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:28.817141 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:28.818474 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:28.819794 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:28.821128 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:28.821817 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:28.821985 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:28.825564 (+ 3579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:28.825571 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:28.825586 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:28.825629 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:28.828471 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:28.829121 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:28.829127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:28.829134 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:28.829289 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:28.829635 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:28.829638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:28.829644 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:28.830176 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:28.830516 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:28.830919 (+ 403us) server_negotiation.cc:299] Negotiation successful
0925 11:03:28.831315 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10991,"thread_start_us":10789,"threads_started":1}
I20260925 11:03:28.833240 5119 log.cc:824] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:28.836418 5119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: No bootstrap required, opened a new log
I20260925 11:03:28.846240 5119 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } }
I20260925 11:03:28.846654 5119 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:28.846750 5119 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb48ab0868d5424f9ad7626c304e89e6, State: Initialized, Role: FOLLOWER
I20260925 11:03:28.847394 5119 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } }
I20260925 11:03:28.847667 5119 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:03:28.847795 5119 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:03:28.847961 5119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:28.850909 5119 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } }
I20260925 11:03:28.851508 5119 leader_election.cc:302] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb48ab0868d5424f9ad7626c304e89e6; no voters:
I20260925 11:03:28.852113 5119 leader_election.cc:288] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:03:28.852200 5124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:03:28.853592 5124 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 1 LEADER]: Becoming Leader. State: Replica: fb48ab0868d5424f9ad7626c304e89e6, State: Running, Role: LEADER
I20260925 11:03:28.854132 5124 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } }
I20260925 11:03:28.854338 5119 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:03:28.857293 5126 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb48ab0868d5424f9ad7626c304e89e6. Latest consensus state: current_term: 1 leader_uuid: "fb48ab0868d5424f9ad7626c304e89e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } } }
I20260925 11:03:28.857651 5126 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:28.858174 5125 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb48ab0868d5424f9ad7626c304e89e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } } }
I20260925 11:03:28.858475 5125 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:28.863679 5131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:03:28.868723 5131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:03:28.892926 5131 catalog_manager.cc:1347] Generated new cluster ID: 49457c00464842e5845e232c3e10cfe9
I20260925 11:03:28.893069 5131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:03:28.911635 5131 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:03:28.913712 5131 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:03:28.925860 5131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Generated new TSK 0
I20260925 11:03:28.927151 5131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:03:28.930107 5119 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:03:29.734165 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:47461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43055
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:03:29 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790334209, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260925 11:03:30.163408 5165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:30.163885 5165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:30.164085 5165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:30.175372 5165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:30.175586 5165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:30.175788 5165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:03:30.191040 5165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.159.212:47461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:30.192812 5165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:30.194571 5165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:30.210592 5175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:30.211961 5176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:30.219826 5178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:30.220625 5165 server_base.cc:1034] running on GCE node
I20260925 11:03:30.221572 5165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:30.227440 5165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:30.228765 5165 hybrid_clock.cc:648] HybridClock initialized: now 1790334210228600 us; error 161 us; skew 500 ppm
I20260925 11:03:30.229224 5165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:30 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334210, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:30.232411 5165 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:03:30.234351 5165 webserver.cc:460] Webserver started at http://127.24.159.193:39979/ using document root <none> and password file <none>
I20260925 11:03:30.235239 5165 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:30.235437 5165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:30.235829 5165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 11:03:30 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334210, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:30.256906 5165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "cedd9b37a1e149fba930aaf6d8fdfe29"
format_stamp: "Formatted at 2026-09-25 11:03:30 on dist-test-slave-trms"
server_key: "097956fa1b2dbd8f0d161cb98222bf02"
server_key_iv: "a65cd466f1d7eed0121ad4c79176a62d"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:30.257915 5165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "cedd9b37a1e149fba930aaf6d8fdfe29"
format_stamp: "Formatted at 2026-09-25 11:03:30 on dist-test-slave-trms"
server_key: "097956fa1b2dbd8f0d161cb98222bf02"
server_key_iv: "a65cd466f1d7eed0121ad4c79176a62d"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:30.264958 5165 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260925 11:03:30.292218 5185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:30.294183 5165 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.002s sys 0.000s
I20260925 11:03:30.294503 5165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "cedd9b37a1e149fba930aaf6d8fdfe29"
format_stamp: "Formatted at 2026-09-25 11:03:30 on dist-test-slave-trms"
server_key: "097956fa1b2dbd8f0d161cb98222bf02"
server_key_iv: "a65cd466f1d7eed0121ad4c79176a62d"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:30.294854 5165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:30.327248 5165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:30.331624 5165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:30.332015 5165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:30.333185 5165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:30.334625 5165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:30.334700 5165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:30.334833 5165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:30.334894 5165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:30.384217 5165 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:43959
I20260925 11:03:30.385929 5165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:03:30.387827 5297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:43959 every 8 connection(s)
I20260925 11:03:30.392469 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 5165
I20260925 11:03:30.392699 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:03:30.393528 25215 external_mini_cluster.cc:1428] Setting key 23537cd0310797a5273c3693a8089528
Sep 25 11:03:30 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334210, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
I20260925 11:03:30.432785 5300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:30.390876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:33169 (local address 127.24.159.254:43055)
0925 11:03:30.401743 (+ 10867us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:30.401750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:30.408696 (+ 6946us) server_negotiation.cc:407] Connection header received
0925 11:03:30.408768 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:30.408773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:30.409781 (+ 1008us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:30.409893 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:30.412102 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:30.413264 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:30.415672 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:30.416326 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:30.416463 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:30.421027 (+ 4564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:30.421034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:30.421039 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:30.421083 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:30.424493 (+ 3410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:30.425540 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:30.425546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:30.425550 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:30.425720 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:30.426342 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:30.426347 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:30.426351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:30.426711 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:30.426856 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:30.429201 (+ 2345us) server_negotiation.cc:299] Negotiation successful
0925 11:03:30.429434 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10716,"thread_start_us":10863,"threads_started":1}
I20260925 11:03:30.434274 5301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:30.391557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43055 (local address 127.24.159.193:33169)
0925 11:03:30.405784 (+ 14227us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:30.405841 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:30.408218 (+ 2377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:30.410362 (+ 2144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:30.410374 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:30.411073 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:30.411900 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:30.411926 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:30.413478 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:30.413483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:30.415508 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:30.415523 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:30.416784 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:30.416792 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:30.417120 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:30.417851 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:30.417888 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:30.420715 (+ 2827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:30.424686 (+ 3971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:30.424695 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:30.424715 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:30.425335 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:30.425862 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:30.425867 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:30.425870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:30.426145 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:30.426926 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:30.426934 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:30.427382 (+ 448us) client_negotiation.cc:769] Sending connection context
0925 11:03:30.427679 (+ 297us) client_negotiation.cc:240] Negotiation successful
0925 11:03:30.428360 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13904,"thread_start_us":17187,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
I20260925 11:03:30.436338 5298 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43055
I20260925 11:03:30.436681 5298 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:30.437568 5298 heartbeater.cc:511] Master 127.24.159.254:43055 requested a full tablet report, sending...
I20260925 11:03:30.440292 5084 ts_manager.cc:194] Registered new tserver with Master: cedd9b37a1e149fba930aaf6d8fdfe29 (127.24.159.193:43959)
I20260925 11:03:30.442422 5084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:33169
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:03:30.469934 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:47461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43055
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:03:30 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334209, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
Sep 25 11:03:30 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334209, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
W20260925 11:03:30.888816 5304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:30.889307 5304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:30.889483 5304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:30.900620 5304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:30.900841 5304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:30.901065 5304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:03:30.916242 5304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.159.212:47461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:30.918164 5304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:30.919981 5304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:30.959041 5312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:30.966878 5311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:30.976415 5315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:30.977056 5304 server_base.cc:1034] running on GCE node
I20260925 11:03:30.977864 5304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:30.983172 5304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:30.984477 5304 hybrid_clock.cc:648] HybridClock initialized: now 1790334210984370 us; error 107 us; skew 500 ppm
I20260925 11:03:30.984920 5304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:30 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334210, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:30.989311 5304 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:03:31.017429 5304 webserver.cc:460] Webserver started at http://127.24.159.194:45129/ using document root <none> and password file <none>
I20260925 11:03:31.018587 5304 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:31.018808 5304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:31.019212 5304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 11:03:31 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334210, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:31.051035 5304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "c87e9bbfed3b4494ac82935569f5868e"
format_stamp: "Formatted at 2026-09-25 11:03:31 on dist-test-slave-trms"
server_key: "381fd0afd48129419172ea921afc681e"
server_key_iv: "271548584059923f39addef146f019ec"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:31.052196 5304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "c87e9bbfed3b4494ac82935569f5868e"
format_stamp: "Formatted at 2026-09-25 11:03:31 on dist-test-slave-trms"
server_key: "381fd0afd48129419172ea921afc681e"
server_key_iv: "271548584059923f39addef146f019ec"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:31.076292 5304 fs_manager.cc:691] Time spent creating directory manager: real 0.023s user 0.011s sys 0.011s
I20260925 11:03:31.133829 5321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:31.135911 5304 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.005s sys 0.002s
I20260925 11:03:31.136211 5304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "c87e9bbfed3b4494ac82935569f5868e"
format_stamp: "Formatted at 2026-09-25 11:03:31 on dist-test-slave-trms"
server_key: "381fd0afd48129419172ea921afc681e"
server_key_iv: "271548584059923f39addef146f019ec"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:31.136533 5304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:31.225677 5304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:31.230813 5304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:31.231133 5304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:31.232321 5304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:31.234092 5304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:31.234182 5304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:31.234336 5304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:31.234400 5304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:31.267432 5304 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:38697
I20260925 11:03:31.267452 5440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:38697 every 8 connection(s)
I20260925 11:03:31.269003 5304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:03:31.274869 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 5304
I20260925 11:03:31.275039 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:03:31.275606 25215 external_mini_cluster.cc:1428] Setting key 1235fa85feab036bbb58c0b830d64234
Sep 25 11:03:31 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334210, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:31.290807 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:31.271514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:53997 (local address 127.24.159.254:43055)
0925 11:03:31.272003 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:31.272010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:31.272970 (+ 960us) server_negotiation.cc:407] Connection header received
0925 11:03:31.274861 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:31.274867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:31.274967 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:31.275084 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:31.276831 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.277691 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.279962 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.280522 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.280665 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:31.284693 (+ 4028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:31.284700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:31.284704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:31.284743 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:31.287611 (+ 2868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:31.288287 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:31.288290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:31.288293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:31.288402 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:31.288788 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:31.288792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:31.288794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:31.289090 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:31.289219 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:31.289898 (+ 679us) server_negotiation.cc:299] Negotiation successful
0925 11:03:31.290440 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":235,"threads_started":1}
I20260925 11:03:31.291555 5444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:31.272076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43055 (local address 127.24.159.194:53997)
0925 11:03:31.272811 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:31.272857 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:31.274551 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:31.275346 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:31.275357 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:31.275927 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:31.276677 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:31.276697 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.277847 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.277853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:31.279804 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:31.279818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.280769 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.280777 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:31.281070 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:31.281831 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:31.281859 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:31.284471 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:31.287733 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:31.287742 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:31.287759 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:31.288170 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:31.288499 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:31.288504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:31.288508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:31.288681 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:31.289212 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:31.289220 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:31.289542 (+ 322us) client_negotiation.cc:769] Sending connection context
0925 11:03:31.289770 (+ 228us) client_negotiation.cc:240] Negotiation successful
0925 11:03:31.290308 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":282,"threads_started":1}
I20260925 11:03:31.293036 5441 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43055
I20260925 11:03:31.293341 5441 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:31.294019 5441 heartbeater.cc:511] Master 127.24.159.254:43055 requested a full tablet report, sending...
I20260925 11:03:31.295409 5084 ts_manager.cc:194] Registered new tserver with Master: c87e9bbfed3b4494ac82935569f5868e (127.24.159.194:38697)
I20260925 11:03:31.296254 5084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:53997
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:03:31.328898 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:47461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43055
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:03:31.447206 5298 heartbeater.cc:503] Master 127.24.159.254:43055 was elected leader, sending a full tablet report...
W20260925 11:03:31.682921 5447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:31.683188 5447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:31.683319 5447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:31.690773 5447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:31.690881 5447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:31.691040 5447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:03:31.700656 5447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.159.212:47461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:43055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:31.701912 5447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:31.703174 5447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:31.715548 5452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:31.715564 5455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:31.715605 5453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:31.716389 5447 server_base.cc:1034] running on GCE node
I20260925 11:03:31.716903 5447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:31.717806 5447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:31.718984 5447 hybrid_clock.cc:648] HybridClock initialized: now 1790334211718919 us; error 60 us; skew 500 ppm
I20260925 11:03:31.719264 5447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:31 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334211, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:31.722163 5447 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:03:31.723543 5447 webserver.cc:460] Webserver started at http://127.24.159.195:36787/ using document root <none> and password file <none>
I20260925 11:03:31.724171 5447 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:31.724262 5447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:31.724495 5447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 25 11:03:31 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334211, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:31.738368 5447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "f9c89d7aaf084662a5a803d936d44d2a"
format_stamp: "Formatted at 2026-09-25 11:03:31 on dist-test-slave-trms"
server_key: "cb350555bc99c3bb4184f86ae203a8b4"
server_key_iv: "e85d25225b8d95b8ad73825b9da1b618"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:31.739152 5447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "f9c89d7aaf084662a5a803d936d44d2a"
format_stamp: "Formatted at 2026-09-25 11:03:31 on dist-test-slave-trms"
server_key: "cb350555bc99c3bb4184f86ae203a8b4"
server_key_iv: "e85d25225b8d95b8ad73825b9da1b618"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:31.744345 5447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:03:31.761051 5462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:31.762857 5447 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260925 11:03:31.763037 5447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "f9c89d7aaf084662a5a803d936d44d2a"
format_stamp: "Formatted at 2026-09-25 11:03:31 on dist-test-slave-trms"
server_key: "cb350555bc99c3bb4184f86ae203a8b4"
server_key_iv: "e85d25225b8d95b8ad73825b9da1b618"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:31.763288 5447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:31.777190 5447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:31.780529 5447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:31.780763 5447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:31.781574 5447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:31.782897 5447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:31.782965 5447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:31.783049 5447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:31.783089 5447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:31.812122 5447 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:38513
I20260925 11:03:31.812140 5574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:38513 every 8 connection(s)
I20260925 11:03:31.813364 5447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:03:31.822476 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 5447
I20260925 11:03:31.822641 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:03:31.823076 25215 external_mini_cluster.cc:1428] Setting key e11f2f7f96b3e9916baed240c829829e
Sep 25 11:03:31 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334211, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:31.831686 5577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:31.815713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:59915 (local address 127.24.159.254:43055)
0925 11:03:31.816152 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:31.816156 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:31.816954 (+ 798us) server_negotiation.cc:407] Connection header received
0925 11:03:31.818773 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:31.818779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:31.818853 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:31.818945 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:31.820310 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.821061 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.822591 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.823048 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.823157 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:31.826514 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:31.826518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:31.826521 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:31.826553 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:31.828819 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:31.829497 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:31.829500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:31.829503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:31.829630 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:31.830036 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:31.830039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:31.830042 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:31.830345 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:31.830473 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:31.831114 (+ 641us) server_negotiation.cc:299] Negotiation successful
0925 11:03:31.831325 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":233,"threads_started":1}
I20260925 11:03:31.832547 5578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:31.816145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43055 (local address 127.24.159.195:59915)
0925 11:03:31.816802 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:31.816842 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:31.818504 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:31.819184 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:31.819192 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:31.819586 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:31.820153 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:31.820170 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.821199 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.821203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:31.822453 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:31.822465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.823262 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.823269 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:31.823480 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:31.824222 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:31.824245 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:31.826313 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:31.828944 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:31.828953 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:31.828971 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:31.829380 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:31.829727 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:31.829732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:31.829736 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:31.829922 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:31.830456 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:31.830464 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:31.830782 (+ 318us) client_negotiation.cc:769] Sending connection context
0925 11:03:31.831004 (+ 222us) client_negotiation.cc:240] Negotiation successful
0925 11:03:31.831492 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":268,"threads_started":1}
I20260925 11:03:31.833796 5575 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43055
I20260925 11:03:31.834048 5575 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:31.834726 5575 heartbeater.cc:511] Master 127.24.159.254:43055 requested a full tablet report, sending...
I20260925 11:03:31.836038 5084 ts_manager.cc:194] Registered new tserver with Master: f9c89d7aaf084662a5a803d936d44d2a (127.24.159.195:38513)
I20260925 11:03:31.836838 5084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:59915
I20260925 11:03:31.838228 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:03:31.839846 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 5165
Sep 25 11:03:31 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:03:31.855098 5580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:31.840741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38848 (local address 127.24.159.193:43959)
0925 11:03:31.841321 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:31.841327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:31.841351 (+ 24us) server_negotiation.cc:407] Connection header received
0925 11:03:31.841407 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:31.841413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:31.841586 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:31.841700 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:31.843137 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.844140 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.845534 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.846101 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.846229 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:31.850232 (+ 4003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:31.850244 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:31.850257 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:31.850290 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:31.852842 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:31.853439 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:31.853445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:31.853448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:31.853588 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:31.853845 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:31.853848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:31.853850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:31.854164 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:31.854415 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:31.854601 (+ 186us) server_negotiation.cc:299] Negotiation successful
0925 11:03:31.854823 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":434,"thread_start_us":275,"threads_started":1}
I20260925 11:03:31.856302 5293 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:31.936403 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 5165
I20260925 11:03:31.949194 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 5304
Sep 25 11:03:31 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:03:31.963809 5582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:31.949966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35516 (local address 127.24.159.194:38697)
0925 11:03:31.950509 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:31.950516 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:31.950541 (+ 25us) server_negotiation.cc:407] Connection header received
0925 11:03:31.950610 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:31.950617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:31.950824 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:31.950948 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:31.952058 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.952994 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.954417 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:31.954970 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:31.955064 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:31.958966 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:31.958977 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:31.958991 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:31.959028 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:31.961535 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:31.962150 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:31.962155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:31.962157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:31.962320 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:31.962620 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:31.962623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:31.962625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:31.962924 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:31.963134 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:31.963345 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 11:03:31.963531 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":297,"threads_started":1}
I20260925 11:03:31.964947 5436 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:32.047703 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 5304
I20260925 11:03:32.060093 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 5447
Sep 25 11:03:32 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:03:32.079717 5584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:32.064920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44324 (local address 127.24.159.195:38513)
0925 11:03:32.065451 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:32.065458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:32.065484 (+ 26us) server_negotiation.cc:407] Connection header received
0925 11:03:32.065542 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:32.065549 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:32.065754 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:32.065943 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:32.067297 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:32.068343 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:32.069663 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:32.070290 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:32.070402 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:32.074501 (+ 4099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:32.074511 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:32.074525 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:32.074564 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:32.077016 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:32.077687 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:32.077694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:32.077697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:32.077820 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:32.078171 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:32.078175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:32.078177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:32.078533 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:32.078833 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:32.079125 (+ 292us) server_negotiation.cc:299] Negotiation successful
0925 11:03:32.079389 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":245,"threads_started":1}
I20260925 11:03:32.080937 5570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:32.160917 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 5447
I20260925 11:03:32.172993 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 5052
I20260925 11:03:32.174129 5114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:32.277762 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 5052
I20260925 11:03:32.294569 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43055
--webserver_interface=127.24.159.254
--webserver_port=44667
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:43055
--ranger_config_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:47461/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.159.212:47461/kms with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:32.607050 5587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:32.607355 5587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:32.607486 5587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:32.614666 5587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:03:32.614794 5587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:32.614871 5587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:32.614945 5587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:03:32.615007 5587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:03:32.625267 5587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.159.212:47461/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:43055
--ranger_config_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=44667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:32.626608 5587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:32.627902 5587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:32.638918 5595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:32.638984 5592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:32.638962 5593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:32.640254 5587 server_base.cc:1034] running on GCE node
I20260925 11:03:32.640957 5587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:32.642390 5587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:32.643587 5587 hybrid_clock.cc:648] HybridClock initialized: now 1790334212643533 us; error 44 us; skew 500 ppm
I20260925 11:03:32.643898 5587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:32 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334212, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:32.646746 5587 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:03:32.648072 5587 webserver.cc:460] Webserver started at http://127.24.159.254:44667/ using document root <none> and password file <none>
I20260925 11:03:32.648708 5587 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:32.648821 5587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 11:03:32 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334212, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:32.714341 5587 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:03:32.718104 5603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:32.719866 5587 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260925 11:03:32.720067 5587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "fb48ab0868d5424f9ad7626c304e89e6"
format_stamp: "Formatted at 2026-09-25 11:03:28 on dist-test-slave-trms"
server_key: "b95ae634397bc82101672e6aac7b0eb5"
server_key_iv: "35a197dc5368308ff0b84508551c7bce"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:32.721383 5587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:32.761710 5587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:32.766319 5587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:32.766623 5587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:32.783205 5587 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:43055
I20260925 11:03:32.783183 5654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:43055 every 8 connection(s)
I20260925 11:03:32.784649 5587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:03:32.791312 5655 sys_catalog.cc:263] Verifying existing consensus state
I20260925 11:03:32.794487 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 5587
I20260925 11:03:32.794636 5655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Bootstrap starting.
I20260925 11:03:32.795492 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:47461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:43959
--local_ip_for_outbound_sockets=127.24.159.193
--tserver_master_addrs=127.24.159.254:43055
--webserver_port=39979
--webserver_interface=127.24.159.193
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.159.212:47461/kms with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:03:32.815760 5655 log.cc:824] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:32.824738 5655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 11:03:32.825235 5655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Bootstrap complete.
I20260925 11:03:32.833894 5655 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } }
I20260925 11:03:32.834753 5655 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb48ab0868d5424f9ad7626c304e89e6, State: Initialized, Role: FOLLOWER
I20260925 11:03:32.835330 5655 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } }
I20260925 11:03:32.835528 5655 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:03:32.835636 5655 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:03:32.835789 5655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 1 FOLLOWER]: Advancing to term 2
I20260925 11:03:32.839383 5655 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } }
I20260925 11:03:32.839849 5655 leader_election.cc:302] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb48ab0868d5424f9ad7626c304e89e6; no voters:
I20260925 11:03:32.840468 5655 leader_election.cc:288] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 11:03:32.840610 5659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 11:03:32.841590 5659 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [term 2 LEADER]: Becoming Leader. State: Replica: fb48ab0868d5424f9ad7626c304e89e6, State: Running, Role: LEADER
I20260925 11:03:32.842245 5659 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } }
I20260925 11:03:32.842717 5655 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:03:32.844628 5660 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fb48ab0868d5424f9ad7626c304e89e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } } }
I20260925 11:03:32.844997 5660 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:32.846517 5664 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:03:32.848517 5661 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb48ab0868d5424f9ad7626c304e89e6. Latest consensus state: current_term: 2 leader_uuid: "fb48ab0868d5424f9ad7626c304e89e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb48ab0868d5424f9ad7626c304e89e6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43055 } } }
I20260925 11:03:32.848825 5661 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:32.852993 5664 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:03:32.854435 5664 catalog_manager.cc:1259] Loaded cluster ID: 49457c00464842e5845e232c3e10cfe9
I20260925 11:03:32.854532 5664 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:03:32.858804 5664 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:03:32.860313 5664 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Loaded TSK: 0
I20260925 11:03:32.861901 5664 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:03:32.866727 5655 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260925 11:03:33.240782 5657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:33.241168 5657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:33.241331 5657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:33.252564 5657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:33.252697 5657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:33.252874 5657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:03:33.268137 5657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.159.212:47461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:43959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=39979
--tserver_master_addrs=127.24.159.254:43055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:33.269932 5657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:33.271781 5657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:33.300213 5693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:33.301927 5692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:33.307248 5695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:33.310065 5657 server_base.cc:1034] running on GCE node
I20260925 11:03:33.310750 5657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:33.311800 5657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:33.313011 5657 hybrid_clock.cc:648] HybridClock initialized: now 1790334213312950 us; error 56 us; skew 500 ppm
I20260925 11:03:33.313326 5657 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:33 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334213, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:33.317407 5657 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:03:33.339707 5657 webserver.cc:460] Webserver started at http://127.24.159.193:39979/ using document root <none> and password file <none>
I20260925 11:03:33.340889 5657 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:33.341044 5657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 11:03:33 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334213, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:33.407887 5657 fs_manager.cc:709] Time spent opening directory manager: real 0.013s user 0.006s sys 0.005s
I20260925 11:03:33.411589 5704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:33.413281 5657 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260925 11:03:33.413470 5657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "cedd9b37a1e149fba930aaf6d8fdfe29"
format_stamp: "Formatted at 2026-09-25 11:03:30 on dist-test-slave-trms"
server_key: "097956fa1b2dbd8f0d161cb98222bf02"
server_key_iv: "a65cd466f1d7eed0121ad4c79176a62d"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:33.414911 5657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:33.455384 5657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:33.460646 5657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:33.460960 5657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:33.462170 5657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:33.463999 5657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:33.464111 5657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:33.464283 5657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:33.464377 5657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:33.518170 5657 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:43959
I20260925 11:03:33.518342 5816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:43959 every 8 connection(s)
I20260925 11:03:33.520030 5657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:03:33.523833 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 5657
I20260925 11:03:33.524649 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:47461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:38697
--local_ip_for_outbound_sockets=127.24.159.194
--tserver_master_addrs=127.24.159.254:43055
--webserver_port=45129
--webserver_interface=127.24.159.194
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.159.212:47461/kms with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:03:33 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334213, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:33.579236 5820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:33.530739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43055 (local address 127.24.159.193:53117)
0925 11:03:33.544329 (+ 13590us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:33.544395 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:33.546686 (+ 2291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:33.558607 (+ 11921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:33.558622 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:33.559307 (+ 685us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:33.560226 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:33.560257 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:33.562808 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:33.562815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:33.564964 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:33.564979 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:33.566492 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:33.566499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:33.566760 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:33.567508 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:33.567537 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:33.570231 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:33.574423 (+ 4192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:33.574433 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:33.574449 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:33.574999 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:33.575532 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:33.575537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:33.575541 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:33.575788 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:33.576788 (+ 1000us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:33.576796 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:33.577178 (+ 382us) client_negotiation.cc:769] Sending connection context
0925 11:03:33.577461 (+ 283us) client_negotiation.cc:240] Negotiation successful
0925 11:03:33.577976 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13246,"thread_start_us":23386,"threads_started":1}
I20260925 11:03:33.579537 5819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:33.530675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:53117 (local address 127.24.159.254:43055)
0925 11:03:33.547044 (+ 16369us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:33.547067 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:33.547125 (+ 58us) server_negotiation.cc:407] Connection header received
0925 11:03:33.547617 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:33.547641 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:33.548187 (+ 546us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:33.548645 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:33.560579 (+ 11934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:33.562613 (+ 2034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:33.565212 (+ 2599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:33.566242 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:33.567226 (+ 984us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:33.570525 (+ 3299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:33.570535 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:33.570555 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:33.570659 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:33.574267 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:33.575173 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:33.575182 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:33.575192 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:33.575401 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:33.575924 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:33.575929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:33.575933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:33.576635 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:33.576819 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:33.577760 (+ 941us) server_negotiation.cc:299] Negotiation successful
0925 11:03:33.578257 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16040,"thread_start_us":18275,"threads_started":1}
I20260925 11:03:33.583196 5817 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43055
I20260925 11:03:33.583501 5817 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:33.584323 5817 heartbeater.cc:511] Master 127.24.159.254:43055 requested a full tablet report, sending...
W20260925 11:03:33.587527 5817 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:43055 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 11:03:33.587740 5817 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:33.588006 5817 heartbeater.cc:511] Master 127.24.159.254:43055 requested a full tablet report, sending...
I20260925 11:03:33.589330 5817 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:33.589596 5817 heartbeater.cc:511] Master 127.24.159.254:43055 requested a full tablet report, sending...
W20260925 11:03:33.590850 5817 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 11:03:33.980389 5821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:33.980821 5821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:33.980980 5821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:33.992780 5821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:33.992920 5821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:33.993153 5821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:03:34.008703 5821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.159.212:47461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:38697
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=45129
--tserver_master_addrs=127.24.159.254:43055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
Sep 25 11:03:34 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790334214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:34.010474 5821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:34.012161 5821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:34.047511 5837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:34.048816 5838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:34.067665 5840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:34.068183 5821 server_base.cc:1034] running on GCE node
I20260925 11:03:34.068981 5821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:34.070286 5821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:34.071543 5821 hybrid_clock.cc:648] HybridClock initialized: now 1790334214071477 us; error 49 us; skew 500 ppm
I20260925 11:03:34.071928 5821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:34 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:34.076011 5821 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:03:34.081986 5821 webserver.cc:460] Webserver started at http://127.24.159.194:45129/ using document root <none> and password file <none>
I20260925 11:03:34.082787 5821 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:34.082947 5821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 11:03:34 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:34.103699 5821 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260925 11:03:34.106564 5849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:34.107959 5821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:03:34.108112 5821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "c87e9bbfed3b4494ac82935569f5868e"
format_stamp: "Formatted at 2026-09-25 11:03:31 on dist-test-slave-trms"
server_key: "381fd0afd48129419172ea921afc681e"
server_key_iv: "271548584059923f39addef146f019ec"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:34.109081 5821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:34.129729 5821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:34.133638 5821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:34.133891 5821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:34.134987 5821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:34.136396 5821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:34.136494 5821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:34.136602 5821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:34.136669 5821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:34.187253 5821 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:38697
I20260925 11:03:34.187294 5961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:38697 every 8 connection(s)
I20260925 11:03:34.189034 5821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:03:34.196022 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 5821
I20260925 11:03:34.196656 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.159.212:47461/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:38513
--local_ip_for_outbound_sockets=127.24.159.195
--tserver_master_addrs=127.24.159.254:43055
--webserver_port=36787
--webserver_interface=127.24.159.195
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:34793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.159.212:47461/kms with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:03:34 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:34.230901 5965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:34.193393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:36549 (local address 127.24.159.254:43055)
0925 11:03:34.208779 (+ 15386us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:34.208787 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:34.210251 (+ 1464us) server_negotiation.cc:407] Connection header received
0925 11:03:34.212259 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:34.212265 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:34.212359 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:34.212435 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:34.214713 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:34.215786 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:34.218180 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:34.218768 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:34.218865 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:34.223256 (+ 4391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:34.223264 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:34.223268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:34.223312 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:34.226947 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:34.227933 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:34.227939 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:34.227943 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:34.228090 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:34.228644 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:34.228650 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:34.228653 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:34.229034 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:34.229171 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:34.230262 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0925 11:03:34.230501 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15223,"thread_start_us":15086,"threads_started":1}
I20260925 11:03:34.233568 5964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:34.192752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43055 (local address 127.24.159.194:36549)
0925 11:03:34.209396 (+ 16644us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:34.209453 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:34.211893 (+ 2440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:34.213005 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:34.213017 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:34.213601 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:34.214499 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:34.214529 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:34.215988 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:34.215993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:34.218008 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:34.218023 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:34.219205 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:34.219210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:34.219474 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:34.220214 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:34.220244 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:34.222959 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:34.227134 (+ 4175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:34.227144 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:34.227165 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:34.227790 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:34.228240 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:34.228245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:34.228249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:34.228497 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:34.229275 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:34.229283 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:34.229760 (+ 477us) client_negotiation.cc:769] Sending connection context
0925 11:03:34.231528 (+ 1768us) client_negotiation.cc:240] Negotiation successful
0925 11:03:34.232224 (+ 696us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16339,"thread_start_us":16117,"threads_started":1}
I20260925 11:03:34.235499 5962 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43055
I20260925 11:03:34.235805 5962 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:34.236605 5962 heartbeater.cc:511] Master 127.24.159.254:43055 requested a full tablet report, sending...
I20260925 11:03:34.238636 5620 ts_manager.cc:194] Registered new tserver with Master: c87e9bbfed3b4494ac82935569f5868e (127.24.159.194:38697)
I20260925 11:03:34.240710 5620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:36549
Sep 25 11:03:34 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:34.591017 5817 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:34.591459 5817 heartbeater.cc:511] Master 127.24.159.254:43055 requested a full tablet report, sending...
I20260925 11:03:34.592754 5620 ts_manager.cc:194] Registered new tserver with Master: cedd9b37a1e149fba930aaf6d8fdfe29 (127.24.159.193:43959)
I20260925 11:03:34.593940 5620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:53117
W20260925 11:03:34.631496 5966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:34.631805 5966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:34.631934 5966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:34.639268 5966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:34.639379 5966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:34.639549 5966 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:03:34.649355 5966 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:34793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.159.212:47461/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:38513
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=36787
--tserver_master_addrs=127.24.159.254:43055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:34.650717 5966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:34.652194 5966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:34.667412 5971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:34.668633 5972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:34.679854 5974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:34.681025 5966 server_base.cc:1034] running on GCE node
I20260925 11:03:34.681958 5966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:34.684046 5966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:34.685600 5966 hybrid_clock.cc:648] HybridClock initialized: now 1790334214685492 us; error 104 us; skew 500 ppm
I20260925 11:03:34.686038 5966 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:34 dist-test-slave-trms krb5kdc[4608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:34.694625 5966 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:03:34.696887 5966 webserver.cc:460] Webserver started at http://127.24.159.195:36787/ using document root <none> and password file <none>
I20260925 11:03:34.697888 5966 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:34.698057 5966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 25 11:03:34 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:34.736316 5966 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.008s sys 0.004s
I20260925 11:03:34.740127 5982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:34.743264 5966 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260925 11:03:34.743445 5966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "f9c89d7aaf084662a5a803d936d44d2a"
format_stamp: "Formatted at 2026-09-25 11:03:31 on dist-test-slave-trms"
server_key: "cb350555bc99c3bb4184f86ae203a8b4"
server_key_iv: "e85d25225b8d95b8ad73825b9da1b618"
server_key_version: "kuduclusterkey@0"
I20260925 11:03:34.744822 5966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:34.766917 5966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
Sep 25 11:03:34 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for HTTP/127.24.159.212@KRBTEST.COM
I20260925 11:03:34.774348 5966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:34.774636 5966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:34.777045 5966 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:34.803759 5966 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:34.803886 5966 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:34.804013 5966 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:34.804086 5966 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:35.014357 5966 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:38513
I20260925 11:03:35.014515 6095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:38513 every 8 connection(s)
I20260925 11:03:35.016278 5966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:03:35.025462 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 5966
Sep 25 11:03:35 dist-test-slave-trms krb5kdc[4608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334214, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:35.049623 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.019873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:43055 (local address 127.24.159.195:57003)
0925 11:03:35.021510 (+ 1637us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.021563 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:35.027795 (+ 6232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:35.028806 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:35.028818 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:35.029983 (+ 1165us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:35.030944 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.030975 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.032690 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.032695 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.034715 (+ 2020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.034731 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.035992 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.036001 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.036691 (+ 690us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.037485 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:35.037515 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:35.040307 (+ 2792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:35.044471 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:35.044480 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:35.044500 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:35.045108 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:35.045543 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:35.045548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:35.045552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:35.045838 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:35.046607 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:35.046615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:35.047093 (+ 478us) client_negotiation.cc:769] Sending connection context
0925 11:03:35.047541 (+ 448us) client_negotiation.cc:240] Negotiation successful
0925 11:03:35.048126 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1321,"thread_start_us":1105,"threads_started":1}
I20260925 11:03:35.050752 6099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.020522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:57003 (local address 127.24.159.254:43055)
0925 11:03:35.025826 (+ 5304us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.025833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.025871 (+ 38us) server_negotiation.cc:407] Connection header received
0925 11:03:35.028277 (+ 2406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.028283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.028380 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.031152 (+ 2772us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:35.031324 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.032497 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.034884 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.035533 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.035652 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.040581 (+ 4929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:35.040588 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:35.040592 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:35.040636 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:35.044296 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.045247 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.045253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.045257 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.045412 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.045982 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.045986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.045989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.046356 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:35.046523 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.050061 (+ 3538us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.050334 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5172,"thread_start_us":5564,"threads_started":1}
I20260925 11:03:35.052771 6096 heartbeater.cc:347] Connected to a master server at 127.24.159.254:43055
I20260925 11:03:35.053143 6096 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:35.054068 6096 heartbeater.cc:511] Master 127.24.159.254:43055 requested a full tablet report, sending...
I20260925 11:03:35.055817 5620 ts_manager.cc:194] Registered new tserver with Master: f9c89d7aaf084662a5a803d936d44d2a (127.24.159.195:38513)
I20260925 11:03:35.056977 5620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:57003
I20260925 11:03:35.059681 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.026557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60950 (local address 127.24.159.254:43055)
0925 11:03:35.036518 (+ 9961us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.036523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.036537 (+ 14us) server_negotiation.cc:407] Connection header received
0925 11:03:35.036576 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.036579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.036662 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.036759 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:35.038944 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.040173 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.042134 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.042884 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.042998 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.046493 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:35.046500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:35.046505 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:35.046545 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:35.053480 (+ 6935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.054788 (+ 1308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.054794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.054798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.054955 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.056485 (+ 1530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.056491 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.056495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.056874 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:35.057222 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.059074 (+ 1852us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.059324 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9859,"thread_start_us":9732,"threads_started":1}
I20260925 11:03:35.062703 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:03:35.081104 6100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.068033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60960 (local address 127.24.159.254:43055)
0925 11:03:35.068232 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.068239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.068743 (+ 504us) server_negotiation.cc:407] Connection header received
0925 11:03:35.068927 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.068933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.069013 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.069094 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:35.070558 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.071222 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.072520 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.072923 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.073030 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.075705 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:35.075710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:35.075713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:35.075746 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:35.078676 (+ 2930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.079335 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.079341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.079344 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.079471 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.079794 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.079799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.079803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.080068 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:35.080306 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.080587 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.080803 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:03:35.086125 5620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:03:35.093451 5620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:03:35.155432 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.107404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:38513 (local address 127.0.0.1:44336)
0925 11:03:35.141785 (+ 34381us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.141843 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:35.142128 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:35.145941 (+ 3813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:35.145946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:35.146000 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:35.147489 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.147510 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.149544 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.149549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.152143 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.152160 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.154363 (+ 2203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.154370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.154614 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.154670 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 11:03:35.154790 (+ 120us) client_negotiation.cc:240] Negotiation successful
0925 11:03:35.154996 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34031,"mutex_wait_us":5,"thread_start_us":36787,"threads_started":1}
I20260925 11:03:35.156734 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.108032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44336 (local address 127.24.159.195:38513)
0925 11:03:35.143065 (+ 35033us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.143074 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.143115 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:03:35.143232 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.143241 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.143562 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.143769 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:35.147718 (+ 3949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.149357 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.152336 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.153870 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.154062 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.156050 (+ 1988us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.156125 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.156297 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34832,"thread_start_us":34681,"threads_started":1}
I20260925 11:03:35.157701 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.108345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38860 (local address 127.24.159.193:43959)
0925 11:03:35.140240 (+ 31895us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.140251 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.144780 (+ 4529us) server_negotiation.cc:407] Connection header received
0925 11:03:35.144905 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.144914 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.145180 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.145364 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:35.147665 (+ 2301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.149357 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.153446 (+ 4089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.154853 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.155038 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.155206 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.157195 (+ 1989us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.157356 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31648,"thread_start_us":32566,"threads_started":1}
I20260925 11:03:35.158062 6118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.107526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.0.0.1:38860)
0925 11:03:35.144493 (+ 36967us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.144514 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:35.144643 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:35.145660 (+ 1017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:35.145669 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:35.145709 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:35.147439 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.147455 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.149526 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.149530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.153261 (+ 3731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.153274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.155369 (+ 2095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.155375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.155554 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.155584 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 11:03:35.155654 (+ 70us) client_negotiation.cc:240] Negotiation successful
0925 11:03:35.155769 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36815,"mutex_wait_us":44,"thread_start_us":37421,"threads_started":1}
I20260925 11:03:35.159152 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.108734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35522 (local address 127.24.159.194:38697)
0925 11:03:35.140674 (+ 31940us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.140683 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.147262 (+ 6579us) server_negotiation.cc:407] Connection header received
0925 11:03:35.147391 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.147398 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.147631 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.147808 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:35.150914 (+ 3106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.152287 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.154729 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.156261 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.156379 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.156549 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.158018 (+ 1469us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.158928 (+ 910us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31772,"thread_start_us":31949,"threads_started":1}
I20260925 11:03:35.159468 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.107768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:38697 (local address 127.0.0.1:35522)
0925 11:03:35.147135 (+ 39367us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.147149 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:35.147259 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:35.150157 (+ 2898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:35.150162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:35.150209 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:35.150691 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.150703 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.152464 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.152468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.154566 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.154581 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.156738 (+ 2157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.156743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.157327 (+ 584us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.157358 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 11:03:35.157443 (+ 85us) client_negotiation.cc:240] Negotiation successful
0925 11:03:35.157568 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39282,"mutex_wait_us":80,"thread_start_us":39102,"threads_started":1}
I20260925 11:03:35.160953 6031 tablet_service.cc:1467] Processing CreateTablet for tablet dd9498ab121c4e0c92fc40537c5c2e18 (DEFAULT_TABLE table=test-table [id=7ed4a2647aaf4ae1b0a7b3d968dc84ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:03:35.161458 5752 tablet_service.cc:1467] Processing CreateTablet for tablet dd9498ab121c4e0c92fc40537c5c2e18 (DEFAULT_TABLE table=test-table [id=7ed4a2647aaf4ae1b0a7b3d968dc84ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:03:35.162478 6031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd9498ab121c4e0c92fc40537c5c2e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:35.162339 5897 tablet_service.cc:1467] Processing CreateTablet for tablet dd9498ab121c4e0c92fc40537c5c2e18 (DEFAULT_TABLE table=test-table [id=7ed4a2647aaf4ae1b0a7b3d968dc84ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:03:35.162850 5752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd9498ab121c4e0c92fc40537c5c2e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:35.163529 5897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd9498ab121c4e0c92fc40537c5c2e18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:35.183869 6125 tablet_bootstrap.cc:492] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: Bootstrap starting.
I20260925 11:03:35.188896 6124 tablet_bootstrap.cc:492] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29: Bootstrap starting.
I20260925 11:03:35.189991 6126 tablet_bootstrap.cc:492] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: Bootstrap starting.
I20260925 11:03:35.193162 6126 tablet_bootstrap.cc:654] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:35.195669 6125 tablet_bootstrap.cc:654] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:35.195861 6126 log.cc:824] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:35.196866 6125 log.cc:824] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:35.197046 6124 tablet_bootstrap.cc:654] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:35.198282 6124 log.cc:824] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:35.207578 6124 tablet_bootstrap.cc:492] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29: No bootstrap required, opened a new log
I20260925 11:03:35.207582 6125 tablet_bootstrap.cc:492] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: No bootstrap required, opened a new log
I20260925 11:03:35.207580 6126 tablet_bootstrap.cc:492] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: No bootstrap required, opened a new log
I20260925 11:03:35.207873 6125 ts_tablet_manager.cc:1397] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: Time spent bootstrapping tablet: real 0.024s user 0.010s sys 0.005s
I20260925 11:03:35.207881 6124 ts_tablet_manager.cc:1397] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.006s
I20260925 11:03:35.208045 6126 ts_tablet_manager.cc:1397] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.011s
I20260925 11:03:35.218287 6125 raft_consensus.cc:348] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } }
I20260925 11:03:35.218286 6124 raft_consensus.cc:348] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } }
I20260925 11:03:35.218755 6124 raft_consensus.cc:374] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:35.218755 6125 raft_consensus.cc:374] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:35.218853 6125 raft_consensus.cc:729] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9c89d7aaf084662a5a803d936d44d2a, State: Initialized, Role: FOLLOWER
I20260925 11:03:35.218853 6124 raft_consensus.cc:729] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cedd9b37a1e149fba930aaf6d8fdfe29, State: Initialized, Role: FOLLOWER
I20260925 11:03:35.219507 6124 consensus_queue.cc:260] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } }
I20260925 11:03:35.219551 6125 consensus_queue.cc:260] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } }
I20260925 11:03:35.220618 6126 raft_consensus.cc:348] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } }
I20260925 11:03:35.221102 6126 raft_consensus.cc:374] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:35.221228 6126 raft_consensus.cc:729] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c87e9bbfed3b4494ac82935569f5868e, State: Initialized, Role: FOLLOWER
I20260925 11:03:35.221917 6126 consensus_queue.cc:260] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } }
I20260925 11:03:35.225970 6126 ts_tablet_manager.cc:1428] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: Time spent starting tablet: real 0.018s user 0.008s sys 0.007s
I20260925 11:03:35.226475 5962 heartbeater.cc:503] Master 127.24.159.254:43055 was elected leader, sending a full tablet report...
I20260925 11:03:35.233866 6096 heartbeater.cc:503] Master 127.24.159.254:43055 was elected leader, sending a full tablet report...
I20260925 11:03:35.234265 6124 ts_tablet_manager.cc:1428] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29: Time spent starting tablet: real 0.026s user 0.017s sys 0.011s
I20260925 11:03:35.234503 6125 ts_tablet_manager.cc:1428] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: Time spent starting tablet: real 0.027s user 0.020s sys 0.008s
I20260925 11:03:35.234849 5817 heartbeater.cc:503] Master 127.24.159.254:43055 was elected leader, sending a full tablet report...
W20260925 11:03:35.277843 6097 tablet.cc:2369] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:03:35.281560 5818 tablet.cc:2369] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:03:35.302980 6130 raft_consensus.cc:482] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:03:35.303311 6130 raft_consensus.cc:504] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } }
I20260925 11:03:35.306738 6130 leader_election.cc:288] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cedd9b37a1e149fba930aaf6d8fdfe29 (127.24.159.193:43959), c87e9bbfed3b4494ac82935569f5868e (127.24.159.194:38697)
I20260925 11:03:35.312556 6098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.305613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.24.159.195:57091)
0925 11:03:35.305971 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.305990 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:35.306099 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:35.306917 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:35.306922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:35.306960 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:35.307136 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.307147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.308493 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.308498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.310747 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.310762 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.311860 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.311866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.312036 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.312061 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 11:03:35.312148 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 11:03:35.312273 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":92}
I20260925 11:03:35.312547 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.306327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:57091 (local address 127.24.159.193:43959)
0925 11:03:35.306520 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.306525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.306543 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:03:35.306608 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.306612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.306678 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.306760 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:35.307305 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.308376 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.310958 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.311722 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.311858 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.311933 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.312192 (+ 259us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.312313 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:03:35.313266 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.306152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:53759 (local address 127.24.159.194:38697)
0925 11:03:35.306358 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.306364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.306840 (+ 476us) server_negotiation.cc:407] Connection header received
0925 11:03:35.307051 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.307055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.307120 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.307191 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:35.307665 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.308337 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.310917 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.312107 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.312243 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.312329 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.312888 (+ 559us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.313022 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 11:03:35.313339 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.305731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:38697 (local address 127.24.159.195:53759)
0925 11:03:35.306745 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.306761 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:35.306928 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:35.307222 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:35.307226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:35.307248 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:35.307527 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.307535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.308464 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.308469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.310747 (+ 2278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:35.310762 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.312476 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.312482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:35.312701 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.312738 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 11:03:35.312852 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 11:03:35.312984 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":106,"thread_start_us":745,"threads_started":1}
I20260925 11:03:35.313918 5772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9498ab121c4e0c92fc40537c5c2e18" candidate_uuid: "f9c89d7aaf084662a5a803d936d44d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" is_pre_election: true
I20260925 11:03:35.314522 5772 raft_consensus.cc:2454] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9c89d7aaf084662a5a803d936d44d2a in term 0.
I20260925 11:03:35.314440 5917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9498ab121c4e0c92fc40537c5c2e18" candidate_uuid: "f9c89d7aaf084662a5a803d936d44d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87e9bbfed3b4494ac82935569f5868e" is_pre_election: true
I20260925 11:03:35.314958 5917 raft_consensus.cc:2454] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9c89d7aaf084662a5a803d936d44d2a in term 0.
I20260925 11:03:35.315239 5985 leader_election.cc:302] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cedd9b37a1e149fba930aaf6d8fdfe29, f9c89d7aaf084662a5a803d936d44d2a; no voters:
I20260925 11:03:35.315611 6130 raft_consensus.cc:2781] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:03:35.315753 6130 raft_consensus.cc:482] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:03:35.315855 6130 raft_consensus.cc:3037] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:35.317890 6130 raft_consensus.cc:504] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } }
I20260925 11:03:35.318476 6130 leader_election.cc:288] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [CANDIDATE]: Term 1 election: Requested vote from peers cedd9b37a1e149fba930aaf6d8fdfe29 (127.24.159.193:43959), c87e9bbfed3b4494ac82935569f5868e (127.24.159.194:38697)
I20260925 11:03:35.318862 5772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9498ab121c4e0c92fc40537c5c2e18" candidate_uuid: "f9c89d7aaf084662a5a803d936d44d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29"
I20260925 11:03:35.318946 5917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd9498ab121c4e0c92fc40537c5c2e18" candidate_uuid: "f9c89d7aaf084662a5a803d936d44d2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c87e9bbfed3b4494ac82935569f5868e"
I20260925 11:03:35.319058 5772 raft_consensus.cc:3037] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:35.319115 5917 raft_consensus.cc:3037] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:35.322124 5917 raft_consensus.cc:2454] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9c89d7aaf084662a5a803d936d44d2a in term 1.
I20260925 11:03:35.322136 5772 raft_consensus.cc:2454] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9c89d7aaf084662a5a803d936d44d2a in term 1.
I20260925 11:03:35.322578 5984 leader_election.cc:302] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c87e9bbfed3b4494ac82935569f5868e, f9c89d7aaf084662a5a803d936d44d2a; no voters:
I20260925 11:03:35.322880 6130 raft_consensus.cc:2781] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:03:35.323355 6130 raft_consensus.cc:686] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 1 LEADER]: Becoming Leader. State: Replica: f9c89d7aaf084662a5a803d936d44d2a, State: Running, Role: LEADER
I20260925 11:03:35.323896 6130 consensus_queue.cc:237] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } }
I20260925 11:03:35.329113 5620 catalog_manager.cc:5495] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a reported cstate change: term changed from 0 to 1, leader changed from <none> to f9c89d7aaf084662a5a803d936d44d2a (127.24.159.195). New cstate: current_term: 1 leader_uuid: "f9c89d7aaf084662a5a803d936d44d2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9c89d7aaf084662a5a803d936d44d2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38513 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 } health_report { overall_health: UNKNOWN } } }
I20260925 11:03:35.355762 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.350529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44344 (local address 127.24.159.195:38513)
0925 11:03:35.350705 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.350712 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.350769 (+ 57us) server_negotiation.cc:407] Connection header received
0925 11:03:35.351025 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.351030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.351136 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.351262 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:03:35.351624 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.352361 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.353691 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.354075 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.354185 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.354716 (+ 531us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:03:35.355224 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:03:35.355351 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.355447 (+ 96us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.355525 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 11:03:35.363905 5917 raft_consensus.cc:1261] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [term 1 FOLLOWER]: Refusing update from remote peer f9c89d7aaf084662a5a803d936d44d2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:03:35.363905 5772 raft_consensus.cc:1261] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29 [term 1 FOLLOWER]: Refusing update from remote peer f9c89d7aaf084662a5a803d936d44d2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:03:35.364995 6130 consensus_queue.cc:1035] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "cedd9b37a1e149fba930aaf6d8fdfe29" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:03:35.365265 6134 consensus_queue.cc:1035] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c87e9bbfed3b4494ac82935569f5868e" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 38697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:03:35.379063 6137 mvcc.cc:205] Tried to move back new op lower bound from 7333208946119405568 to 7333208945971191808. Current Snapshot: MvccSnapshot[applied={T|T < 7333208946119405568}]
I20260925 11:03:35.390448 6139 mvcc.cc:205] Tried to move back new op lower bound from 7333208946119405568 to 7333208945971191808. Current Snapshot: MvccSnapshot[applied={T|T < 7333208946119405568}]
I20260925 11:03:35.395922 5619 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:03:35.396291 5619 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:03:35.401181 5619 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb48ab0868d5424f9ad7626c304e89e6: Sending DeleteTablet for 3 replicas of tablet dd9498ab121c4e0c92fc40537c5c2e18
I20260925 11:03:35.402212 5752 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9498ab121c4e0c92fc40537c5c2e18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:35 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:38860
I20260925 11:03:35.402493 5897 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9498ab121c4e0c92fc40537c5c2e18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:35 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35522
I20260925 11:03:35.402536 6031 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9498ab121c4e0c92fc40537c5c2e18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:35 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:44336
I20260925 11:03:35.403273 5896 tablet_service.cc:1514] Processing DeleteTablet for tablet dd9498ab121c4e0c92fc40537c5c2e18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:35 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35522
I20260925 11:03:35.411330 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 5657
I20260925 11:03:35.414999 6148 tablet_replica.cc:330] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: stopping tablet replica
I20260925 11:03:35.415657 6148 raft_consensus.cc:2229] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:03:35.416400 6148 raft_consensus.cc:2258] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:03:35.418753 6148 ts_tablet_manager.cc:1905] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:03:35.425997 6148 ts_tablet_manager.cc:1918] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:03:35.426265 6148 log.cc:1196] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/dd9498ab121c4e0c92fc40537c5c2e18
I20260925 11:03:35.426899 6148 ts_tablet_manager.cc:1939] T dd9498ab121c4e0c92fc40537c5c2e18 P f9c89d7aaf084662a5a803d936d44d2a: Deleting consensus metadata
I20260925 11:03:35.428823 5606 catalog_manager.cc:4841] TS f9c89d7aaf084662a5a803d936d44d2a (127.24.159.195:38513): tablet dd9498ab121c4e0c92fc40537c5c2e18 (table test-table [id=7ed4a2647aaf4ae1b0a7b3d968dc84ef]) successfully deleted
I20260925 11:03:35.429128 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.412835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38876 (local address 127.24.159.193:43959)
0925 11:03:35.412995 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.413001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.413020 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:03:35.413092 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.413097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.413170 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.413266 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:35.414999 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.416135 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.417787 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.418484 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.418605 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.422009 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:35.422027 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:35.422045 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:35.422104 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:35.425844 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.426721 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.426731 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.426735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.426897 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.427266 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.427271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.427274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.427692 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:35.427982 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.428457 (+ 475us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.428679 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 11:03:35.429701 5812 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:35.430564 6147 tablet_replica.cc:330] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: stopping tablet replica
I20260925 11:03:35.431038 6147 raft_consensus.cc:2229] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:03:35.431519 6147 raft_consensus.cc:2258] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:03:35.436211 6147 ts_tablet_manager.cc:1905] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:03:35.443254 6147 ts_tablet_manager.cc:1918] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:03:35.443570 6147 log.cc:1196] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/dd9498ab121c4e0c92fc40537c5c2e18
I20260925 11:03:35.444154 6147 ts_tablet_manager.cc:1939] T dd9498ab121c4e0c92fc40537c5c2e18 P c87e9bbfed3b4494ac82935569f5868e: Deleting consensus metadata
I20260925 11:03:35.446158 5604 catalog_manager.cc:4841] TS c87e9bbfed3b4494ac82935569f5868e (127.24.159.194:38697): tablet dd9498ab121c4e0c92fc40537c5c2e18 (table test-table [id=7ed4a2647aaf4ae1b0a7b3d968dc84ef]) successfully deleted
W20260925 11:03:35.447304 5604 catalog_manager.cc:4808] TS c87e9bbfed3b4494ac82935569f5868e (127.24.159.194:38697): delete failed for tablet dd9498ab121c4e0c92fc40537c5c2e18 because the tablet was not found. No further retry: Not found: Tablet not found: dd9498ab121c4e0c92fc40537c5c2e18
I20260925 11:03:35.539867 6146 tablet_replica.cc:330] T dd9498ab121c4e0c92fc40537c5c2e18 P cedd9b37a1e149fba930aaf6d8fdfe29: stopping tablet replica
I20260925 11:03:35.539960 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 5657
I20260925 11:03:35.553807 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 5821
W20260925 11:03:35.553874 5605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.159.193:43959 (error 108)
I20260925 11:03:35.555750 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.555276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.0.0.1:38882)
0925 11:03:35.555460 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.555582 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260925 11:03:35.556005 5605 catalog_manager.cc:4584] TS cedd9b37a1e149fba930aaf6d8fdfe29 (127.24.159.193:43959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dd9498ab121c4e0c92fc40537c5c2e18: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43959: connect: Connection refused (error 111)
I20260925 11:03:35.567747 6123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.554954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35534 (local address 127.24.159.194:38697)
0925 11:03:35.555161 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.555167 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.555186 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:03:35.555247 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.555252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.555327 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.555418 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:35.556816 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.557551 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.558958 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.559343 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.559447 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.561822 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:35.561835 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:35.561850 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:35.561888 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:35.565017 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.565644 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.565653 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.565657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.565796 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.566138 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.566143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.566146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.566637 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:35.566897 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.567215 (+ 318us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.567417 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:03:35.568444 5957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:35.609704 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.609136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.0.0.1:38896)
0925 11:03:35.609375 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.609511 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260925 11:03:35.610877 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.610485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.0.0.1:38898)
0925 11:03:35.610631 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.610731 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 11:03:35.655766 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 5821
I20260925 11:03:35.669585 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 5966
I20260925 11:03:35.671486 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.671051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.0.0.1:38908)
0925 11:03:35.671221 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.671316 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 11:03:35.672595 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.672192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.0.0.1:38918)
0925 11:03:35.672352 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.672448 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 11:03:35.683486 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.670528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44352 (local address 127.24.159.195:38513)
0925 11:03:35.670724 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:35.670730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:35.670748 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:03:35.670805 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:35.670809 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:35.670869 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:35.670955 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:35.672551 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.673405 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.674747 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:35.675176 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:35.675287 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:35.677839 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:35.677850 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:35.677865 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:35.677896 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:35.680779 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.681422 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.681429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.681431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.681596 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:35.681943 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:35.681946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:35.681948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:35.682358 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:35.682652 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:35.682938 (+ 286us) server_negotiation.cc:299] Negotiation successful
0925 11:03:35.683147 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 11:03:35.684268 6091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:35.753971 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.753411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.0.0.1:38926)
0925 11:03:35.753654 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.753769 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260925 11:03:35.755044 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.754634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.0.0.1:38934)
0925 11:03:35.754830 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.754917 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260925 11:03:35.804349 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 5966
I20260925 11:03:35.820225 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 5587
I20260925 11:03:35.821347 5650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:35.944440 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:35.943605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43959 (local address 127.0.0.1:38940)
0925 11:03:35.943784 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:35.943931 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260925 11:03:35.944741 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 5587
I20260925 11:03:35.962512 25215 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260925 11:03:36.297097 25215 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260925 11:03:36.297291 25215 mini_ranger.cc:66] Stopping Ranger...
I20260925 11:03:36.631378 25215 mini_ranger.cc:68] Stopped Ranger
2026-09-25 11:03:36.631 UTC [4638] LOG: received smart shutdown request
2026-09-25 11:03:36.634 UTC [4638] LOG: background worker "logical replication launcher" (PID 4647) exited with exit code 1
2026-09-25 11:03:36.634 UTC [4642] LOG: shutting down
2026-09-25 11:03:36.771 UTC [4638] LOG: database system is shut down
2026-09-25T11:03:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50550 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:03:36 dist-test-slave-trms krb5kdc[6162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:03:36 dist-test-slave-trms krb5kdc[6162](info): set up 2 sockets
Sep 25 11:03:36 dist-test-slave-trms krb5kdc[6162](info): commencing operation
krb5kdc: starting...
W20260925 11:03:37.870488 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:03:38 dist-test-slave-trms krb5kdc[6162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:03:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:03:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:03:38.084895 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:35963
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:35963,127.24.159.253:43811,127.24.159.252:36097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:38.401693 6175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:38.401984 6175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:38.402114 6175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:38.409324 6175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:03:38.409435 6175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:38.409504 6175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:38.409590 6175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:03:38.409646 6175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:03:38.419454 6175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:35963,127.24.159.253:43811,127.24.159.252:36097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:35963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:38.420786 6175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:38.422108 6175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:38.433398 6181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:38.433494 6180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:38.433372 6183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:38.434160 6175 server_base.cc:1034] running on GCE node
I20260925 11:03:38.434973 6175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:38.436357 6175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:38.437564 6175 hybrid_clock.cc:648] HybridClock initialized: now 1790334218437520 us; error 34 us; skew 500 ppm
I20260925 11:03:38.437894 6175 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:38 dist-test-slave-trms krb5kdc[6162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:38.440917 6175 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:03:38.442355 6175 webserver.cc:460] Webserver started at http://127.24.159.254:35043/ using document root <none> and password file <none>
I20260925 11:03:38.443018 6175 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:38.443131 6175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:38.443415 6175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:03:38.446380 6175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "e000149c6f93411092117255ae9fc0ea"
format_stamp: "Formatted at 2026-09-25 11:03:38 on dist-test-slave-trms"
server_key: "327fdc437fcd6b37e630cc33bbf3eb01"
server_key_iv: "375da0952e238481adbe76b1957f4737"
server_key_version: "encryptionkey@0"
I20260925 11:03:38.447161 6175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "e000149c6f93411092117255ae9fc0ea"
format_stamp: "Formatted at 2026-09-25 11:03:38 on dist-test-slave-trms"
server_key: "327fdc437fcd6b37e630cc33bbf3eb01"
server_key_iv: "375da0952e238481adbe76b1957f4737"
server_key_version: "encryptionkey@0"
I20260925 11:03:38.452237 6175 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:03:38.455619 6190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:38.456938 6175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:03:38.457098 6175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "e000149c6f93411092117255ae9fc0ea"
format_stamp: "Formatted at 2026-09-25 11:03:38 on dist-test-slave-trms"
server_key: "327fdc437fcd6b37e630cc33bbf3eb01"
server_key_iv: "375da0952e238481adbe76b1957f4737"
server_key_version: "encryptionkey@0"
I20260925 11:03:38.457294 6175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:38.469679 6175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:38.473496 6175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:38.473764 6175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:38.489503 6175 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:35963
I20260925 11:03:38.489531 6241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:35963 every 8 connection(s)
I20260925 11:03:38.490873 6175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:03:38.493191 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 6175
I20260925 11:03:38.493409 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:03:38.494179 25215 external_mini_cluster.cc:1428] Setting key 1855f66955e7411dcc1ae61991d9c12b
I20260925 11:03:38.495172 6242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:38.503542 6242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.159.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.253@KRBTEST.COM" created.
Sep 25 11:03:38 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:38.526741 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.506656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:35963 (local address 127.0.0.1:52234)
0925 11:03:38.507580 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.507659 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:38.509983 (+ 2324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:38.510851 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:38.510858 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:38.511604 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:38.512068 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:38.512096 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.513583 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.513589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:38.515391 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:38.515407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.516339 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.516349 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:38.516530 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:38.517141 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:38.517163 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:38.519828 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:38.522774 (+ 2946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:38.522786 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:38.522798 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:38.523184 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:38.523555 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:38.523559 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:38.523563 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:38.523711 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:38.524428 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:38.524437 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:38.524638 (+ 201us) client_negotiation.cc:769] Sending connection context
0925 11:03:38.524842 (+ 204us) client_negotiation.cc:240] Negotiation successful
0925 11:03:38.525422 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":116,"thread_start_us":391,"threads_started":1}
I20260925 11:03:38.526758 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.506656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.24.159.254:35963)
0925 11:03:38.508452 (+ 1796us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:38.508458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:38.508498 (+ 40us) server_negotiation.cc:407] Connection header received
0925 11:03:38.510378 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:38.510383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:38.510623 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:38.510773 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:38.512259 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.513417 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.515525 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.516081 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.516206 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:38.520036 (+ 3830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:38.520044 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:38.520056 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:38.520084 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:38.522661 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:38.523308 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:38.523315 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:38.523317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:38.523454 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:38.523812 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:38.523815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:38.523817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:38.524304 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:38.524489 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:38.524995 (+ 506us) server_negotiation.cc:299] Negotiation successful
0925 11:03:38.525270 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1606,"spinlock_wait_cycles":16640,"thread_start_us":1368,"threads_started":1}
I20260925 11:03:38.529809 6242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:03:38.530922 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.530553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36528)
0925 11:03:38.530707 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.530810 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260925 11:03:38.531184 6191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
I20260925 11:03:38.532631 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.532302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36538)
0925 11:03:38.532436 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.532500 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 11:03:38.532802 6242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:43811: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab.
Entry for principal kudu/127.24.159.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab.
I20260925 11:03:38.543537 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.253:43811
--webserver_interface=127.24.159.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab
--principal=kudu/127.24.159.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:35963,127.24.159.253:43811,127.24.159.252:36097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:03:38.582155 6242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } attempt: 1
I20260925 11:03:38.583302 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.582854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36550)
0925 11:03:38.583054 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.583173 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 11:03:38.584321 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.583992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36554)
0925 11:03:38.584119 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.584189 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 11:03:38.584520 6242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:43811: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
I20260925 11:03:38.652766 6242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } attempt: 2
I20260925 11:03:38.653959 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.653488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36566)
0925 11:03:38.653725 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.653817 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260925 11:03:38.654963 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.654603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36570)
0925 11:03:38.654759 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.654827 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260925 11:03:38.655156 6242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:43811: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
I20260925 11:03:38.746347 6242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } attempt: 3
I20260925 11:03:38.747470 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.747027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36578)
0925 11:03:38.747228 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.747333 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260925 11:03:38.748562 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.748215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36582)
0925 11:03:38.748340 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.748412 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 11:03:38.748752 6242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:43811: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
W20260925 11:03:38.859725 6249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:38.860000 6249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:38.860090 6249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:38.867233 6249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:03:38.867331 6249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:38.867381 6249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:38.867435 6249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:03:38.867476 6249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:03:38.876906 6249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:35963,127.24.159.253:43811,127.24.159.252:36097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab
--principal=kudu/127.24.159.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.253:43811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.159.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:38.878082 6249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:38.879371 6249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:38.890239 6255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:38.890219 6257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:38.890280 6254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:38.890982 6249 server_base.cc:1034] running on GCE node
I20260925 11:03:38.891686 6249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:38.891948 6242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } attempt: 4
I20260925 11:03:38.892998 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.892517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36592)
0925 11:03:38.892790 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.892869 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
I20260925 11:03:38.893460 6249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:38.893970 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.893615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36608)
0925 11:03:38.893762 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.893849 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 11:03:38.894137 6242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:43811: Network error: Client connection negotiation failed: client connection to 127.24.159.253:43811: connect: Connection refused (error 111)
I20260925 11:03:38.894692 6249 hybrid_clock.cc:648] HybridClock initialized: now 1790334218894632 us; error 50 us; skew 500 ppm
I20260925 11:03:38.895025 6249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:38 dist-test-slave-trms krb5kdc[6162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:38.897871 6249 init.cc:377] Logged in from keytab as kudu/127.24.159.253@KRBTEST.COM (short username kudu)
I20260925 11:03:38.899212 6249 webserver.cc:460] Webserver started at http://127.24.159.253:36885/ using document root <none> and password file <none>
I20260925 11:03:38.899858 6249 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:38.899972 6249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:38.900254 6249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:03:38.903127 6249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/data/instance:
uuid: "6ec0c40d1b134b9397c961358e1b44f8"
format_stamp: "Formatted at 2026-09-25 11:03:38 on dist-test-slave-trms"
server_key: "25622f4e1c13e5ed9b9790d3dbe05d79"
server_key_iv: "c15178a24e303b8b2ec45efb666f50a7"
server_key_version: "encryptionkey@0"
I20260925 11:03:38.903879 6249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/wal/instance:
uuid: "6ec0c40d1b134b9397c961358e1b44f8"
format_stamp: "Formatted at 2026-09-25 11:03:38 on dist-test-slave-trms"
server_key: "25622f4e1c13e5ed9b9790d3dbe05d79"
server_key_iv: "c15178a24e303b8b2ec45efb666f50a7"
server_key_version: "encryptionkey@0"
I20260925 11:03:38.908962 6249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:03:38.912526 6264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:38.913867 6249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 11:03:38.914026 6249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/wal
uuid: "6ec0c40d1b134b9397c961358e1b44f8"
format_stamp: "Formatted at 2026-09-25 11:03:38 on dist-test-slave-trms"
server_key: "25622f4e1c13e5ed9b9790d3dbe05d79"
server_key_iv: "c15178a24e303b8b2ec45efb666f50a7"
server_key_version: "encryptionkey@0"
I20260925 11:03:38.914238 6249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:38.936028 6249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:38.939968 6249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:38.940234 6249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:38.956399 6249 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.253:43811
I20260925 11:03:38.956429 6315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.253:43811 every 8 connection(s)
I20260925 11:03:38.957679 6249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
I20260925 11:03:38.961210 6316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:38.963148 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 6249
I20260925 11:03:38.963297 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-1/wal/instance
I20260925 11:03:38.963687 25215 external_mini_cluster.cc:1428] Setting key 0f4805643639cfc7b1bdbaf9f1ca7753
I20260925 11:03:38.967581 6316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:03:38 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.159.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.252@KRBTEST.COM" created.
I20260925 11:03:38.987053 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.970140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.24.159.254:35963)
0925 11:03:38.970315 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:38.970319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:38.971741 (+ 1422us) server_negotiation.cc:407] Connection header received
0925 11:03:38.973550 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:38.973557 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:38.973644 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:38.973747 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:38.975251 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.976197 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.977679 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.978359 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.978496 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:38.981655 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:38.981662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:38.981667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:38.981710 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:38.984177 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:38.984982 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:38.984986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:38.984988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:38.985103 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:38.985505 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:38.985507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:38.985509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:38.985768 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:38.985874 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:38.986656 (+ 782us) server_negotiation.cc:299] Negotiation successful
0925 11:03:38.986823 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:03:38.988396 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.970782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:35963 (local address 127.0.0.1:52238)
0925 11:03:38.971562 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.971612 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:38.973283 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:38.973938 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:38.973946 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:38.974506 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:38.975064 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:38.975081 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.976344 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.976348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:38.977545 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:38.977557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.978581 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.978587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:38.978775 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:38.979381 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:38.979403 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:38.981424 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:38.984304 (+ 2880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:38.984311 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:38.984323 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:38.984884 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:38.985204 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:38.985208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:38.985212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:38.985419 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:38.985874 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:38.985883 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:38.986304 (+ 421us) client_negotiation.cc:769] Sending connection context
0925 11:03:38.986591 (+ 287us) client_negotiation.cc:240] Negotiation successful
0925 11:03:38.987058 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":312,"threads_started":1}
I20260925 11:03:38.990113 6316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:03:38 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
I20260925 11:03:39.004581 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.990847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36618)
0925 11:03:38.990999 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:38.991013 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:38.991090 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:38.992224 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:38.992227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:38.992546 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:38.992701 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:38.992712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.993952 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.993956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:38.995270 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:38.995284 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.996043 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.996052 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:38.996262 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:38.996794 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:38.996815 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:38.999041 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:39.001989 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.001995 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.001999 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.002526 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.002861 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.002866 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.002869 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.003042 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.003619 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:39.003624 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:39.003774 (+ 150us) client_negotiation.cc:769] Sending connection context
0925 11:03:39.003920 (+ 146us) client_negotiation.cc:240] Negotiation successful
0925 11:03:39.004201 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 11:03:39.004596 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:38.991108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.24.159.253:43811)
0925 11:03:38.991732 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:38.991739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:38.991769 (+ 30us) server_negotiation.cc:407] Connection header received
0925 11:03:38.991817 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:38.991824 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:38.992052 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:38.992201 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:38.992844 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.993820 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.995392 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:38.995906 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:38.995998 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:38.999240 (+ 3242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:38.999251 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:38.999265 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:38.999301 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.001854 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.002638 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.002644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.002647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.002761 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.003150 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.003153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.003155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.003488 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.003647 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.004035 (+ 388us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.004302 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":458,"thread_start_us":313,"threads_started":1}
I20260925 11:03:39.007102 6316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:03:39.008157 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.007682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:50954)
0925 11:03:39.007855 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.007973 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260925 11:03:39.008369 6265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
I20260925 11:03:39.009769 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.009442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:50970)
0925 11:03:39.009598 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.009670 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 11:03:39.009992 6316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:36097: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab.
Entry for principal kudu/127.24.159.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab.
I20260925 11:03:39.011669 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.252:36097
--webserver_interface=127.24.159.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab
--principal=kudu/127.24.159.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:35963,127.24.159.253:43811,127.24.159.252:36097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:03:39.059275 6316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } attempt: 1
I20260925 11:03:39.060441 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.059980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:50984)
0925 11:03:39.060187 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.060288 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260925 11:03:39.061596 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.061104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:50992)
0925 11:03:39.061348 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.061439 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260925 11:03:39.061931 6316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:36097: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
I20260925 11:03:39.130167 6316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } attempt: 2
I20260925 11:03:39.131408 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.130843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51002)
0925 11:03:39.131094 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.131214 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260925 11:03:39.132473 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.132058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51004)
0925 11:03:39.132218 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.132308 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 11:03:39.132668 6316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:36097: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
I20260925 11:03:39.193409 6242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } attempt: 5
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
I20260925 11:03:39.206354 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.194183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36632)
0925 11:03:39.194431 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.194452 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:39.194593 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:39.194952 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:39.194957 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:39.195338 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:39.195520 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.195530 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.196512 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.196518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.197399 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.197412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.198115 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.198121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.198319 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.198936 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:39.198953 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:39.201150 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:39.204026 (+ 2876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.204032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.204036 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.204394 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.204743 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.204748 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.204752 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.204884 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.205394 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:39.205398 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:39.205547 (+ 149us) client_negotiation.cc:769] Sending connection context
0925 11:03:39.205678 (+ 131us) client_negotiation.cc:240] Negotiation successful
0925 11:03:39.205954 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260925 11:03:39.206353 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.194253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.24.159.253:43811)
0925 11:03:39.194431 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.194437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.194530 (+ 93us) server_negotiation.cc:407] Connection header received
0925 11:03:39.194732 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.194736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.194813 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.194900 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.195694 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.196380 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.197549 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.197961 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.198069 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.201350 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.201355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.201358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.201393 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.203893 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.204508 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.204512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.204514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.204638 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.204981 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.204986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.204989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.205281 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.205412 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.205788 (+ 376us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.206001 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 11:03:39.207713 6242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:03:39.208807 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.208328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51020)
0925 11:03:39.208529 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.208631 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 11:03:39.209780 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.209415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51028)
0925 11:03:39.209572 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.209651 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 11:03:39.209969 6242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:36097: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
I20260925 11:03:39.223870 6316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } attempt: 3
I20260925 11:03:39.224947 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.224422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51038)
0925 11:03:39.224630 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.224746 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 11:03:39.226022 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.225562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51044)
0925 11:03:39.225788 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.225881 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260925 11:03:39.226260 6316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:36097: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
I20260925 11:03:39.261196 6242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } attempt: 1
I20260925 11:03:39.262392 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.261848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51046)
0925 11:03:39.262052 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.262170 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 11:03:39.263348 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.262993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51048)
0925 11:03:39.263135 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.263219 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 11:03:39.263568 6242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:36097: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
W20260925 11:03:39.329725 6323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:39.330030 6323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:39.330170 6323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260925 11:03:39.331785 6242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } attempt: 2
I20260925 11:03:39.332922 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.332358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51062)
0925 11:03:39.332617 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.332728 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260925 11:03:39.333941 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.333579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51068)
0925 11:03:39.333726 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.333812 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 11:03:39.334172 6242 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:36097: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
W20260925 11:03:39.338013 6323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:03:39.338112 6323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:39.338164 6323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:39.338264 6323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:03:39.338326 6323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:03:39.348090 6323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:35963,127.24.159.253:43811,127.24.159.252:36097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab
--principal=kudu/127.24.159.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.252:36097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.159.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:39.349473 6323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:39.350899 6323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:39.362534 6329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:39.362571 6328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:39.362500 6331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:39.363315 6323 server_base.cc:1034] running on GCE node
I20260925 11:03:39.364095 6323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:39.365500 6323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:39.366685 6323 hybrid_clock.cc:648] HybridClock initialized: now 1790334219366641 us; error 41 us; skew 500 ppm
I20260925 11:03:39.367022 6323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334219, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:39.369483 6316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } attempt: 4
I20260925 11:03:39.370141 6323 init.cc:377] Logged in from keytab as kudu/127.24.159.252@KRBTEST.COM (short username kudu)
I20260925 11:03:39.370565 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.370061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51080)
0925 11:03:39.370308 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.370404 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 11:03:39.371605 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.371269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51090)
0925 11:03:39.371415 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.371497 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260925 11:03:39.371829 6316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:36097: Network error: Client connection negotiation failed: client connection to 127.24.159.252:36097: connect: Connection refused (error 111)
I20260925 11:03:39.372272 6323 webserver.cc:460] Webserver started at http://127.24.159.252:46663/ using document root <none> and password file <none>
I20260925 11:03:39.372952 6323 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:39.373054 6323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:39.373309 6323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:03:39.376360 6323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/data/instance:
uuid: "7e5967a8b4a7405ba61cc3d1013eb11a"
format_stamp: "Formatted at 2026-09-25 11:03:39 on dist-test-slave-trms"
server_key: "7bef25829a88b6f6eeae6c787912d08e"
server_key_iv: "99f59e3647f8a4ae6217d91c335cc9e4"
server_key_version: "encryptionkey@0"
I20260925 11:03:39.377131 6323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/wal/instance:
uuid: "7e5967a8b4a7405ba61cc3d1013eb11a"
format_stamp: "Formatted at 2026-09-25 11:03:39 on dist-test-slave-trms"
server_key: "7bef25829a88b6f6eeae6c787912d08e"
server_key_iv: "99f59e3647f8a4ae6217d91c335cc9e4"
server_key_version: "encryptionkey@0"
I20260925 11:03:39.382352 6323 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260925 11:03:39.385681 6338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:39.387018 6323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:03:39.387159 6323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/wal
uuid: "7e5967a8b4a7405ba61cc3d1013eb11a"
format_stamp: "Formatted at 2026-09-25 11:03:39 on dist-test-slave-trms"
server_key: "7bef25829a88b6f6eeae6c787912d08e"
server_key_iv: "99f59e3647f8a4ae6217d91c335cc9e4"
server_key_version: "encryptionkey@0"
I20260925 11:03:39.387316 6323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:39.405925 6323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:39.409869 6323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:39.410100 6323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:39.426857 6323 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.252:36097
I20260925 11:03:39.426891 6389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.252:36097 every 8 connection(s)
I20260925 11:03:39.428196 6323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
I20260925 11:03:39.431025 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 6323
I20260925 11:03:39.431212 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790334022505657-25215-0/minicluster-data/master-2/wal/instance
I20260925 11:03:39.431766 25215 external_mini_cluster.cc:1428] Setting key 51c50fa8b0a29cdcc48446525338faa4
I20260925 11:03:39.432273 6390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:39.440421 6242 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } attempt: 3
I20260925 11:03:39.441402 6390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334219, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:39.456614 6245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.441005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51110)
0925 11:03:39.441207 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.441223 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:39.441318 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:39.442063 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:39.442068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:39.442500 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:39.442689 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.442699 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.444094 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.444100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.445617 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.445637 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.446721 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.446729 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.446929 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.447564 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:39.447584 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:39.449748 (+ 2164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:39.453869 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.453875 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.453878 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.454364 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.454770 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.454774 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.454778 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.454958 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.455547 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:39.455552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:39.455699 (+ 147us) client_negotiation.cc:769] Sending connection context
0925 11:03:39.455828 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 11:03:39.456181 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 11:03:39.458034 6394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.441145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.24.159.252:36097)
0925 11:03:39.441794 (+ 649us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.441798 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.441814 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:03:39.441849 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.441852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.441904 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.441990 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.442838 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.443932 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.445789 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.446535 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.447229 (+ 694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.449967 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.449973 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.449978 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.450016 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.453739 (+ 3723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.454498 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.454503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.454506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.454653 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.455087 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.455092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.455095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.455405 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.456168 (+ 763us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.456444 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.456868 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":381,"threads_started":1}
I20260925 11:03:39.458065 6393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.433640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.24.159.252:36097)
0925 11:03:39.434440 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.434452 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.434492 (+ 40us) server_negotiation.cc:407] Connection header received
0925 11:03:39.436742 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.436765 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.437193 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.437526 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.439266 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.440633 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.443229 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.444258 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.444429 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.449371 (+ 4942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.449381 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.449396 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.449438 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.453128 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.453824 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.453834 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.453842 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.454088 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.454572 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.454578 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.454581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.455295 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.455586 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.456517 (+ 931us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.456975 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":357,"threads_started":1}
I20260925 11:03:39.463536 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.444387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:35963 (local address 127.0.0.1:52250)
0925 11:03:39.446334 (+ 1947us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.446378 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:39.446582 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:39.447007 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:39.447013 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:39.447801 (+ 788us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:39.448112 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.448124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.449484 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.449491 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.451053 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.451067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.452199 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.452206 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.452403 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.453057 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:39.453081 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:39.455649 (+ 2568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:39.460477 (+ 4828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.460487 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.460495 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.461033 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.461464 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.461469 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.461474 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.461635 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.462442 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:39.462450 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:39.462624 (+ 174us) client_negotiation.cc:769] Sending connection context
0925 11:03:39.462817 (+ 193us) client_negotiation.cc:240] Negotiation successful
0925 11:03:39.463136 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1674,"spinlock_wait_cycles":20224,"thread_start_us":1543,"threads_started":1}
I20260925 11:03:39.463639 6246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.445059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52250 (local address 127.24.159.254:35963)
0925 11:03:39.445250 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.445254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.446458 (+ 1204us) server_negotiation.cc:407] Connection header received
0925 11:03:39.446715 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.446719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.446777 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.446825 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.448283 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.449330 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.451253 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.451887 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.452010 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.455844 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.455849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.455853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.455885 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.460325 (+ 4440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.461150 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.461154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.461157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.461311 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.461756 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.461761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.461765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.462171 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.462364 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.462983 (+ 619us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.463213 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":68}
I20260925 11:03:39.466141 6242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea: Bootstrap starting.
I20260925 11:03:39.469573 6242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea: Neither blocks nor log segments found. Creating new log.
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:39.470788 6242 log.cc:824] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:39.471007 6390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:03:39.474367 6242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea: No bootstrap required, opened a new log
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334219, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
I20260925 11:03:39.478760 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.461233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52266 (local address 127.24.159.254:35963)
0925 11:03:39.462667 (+ 1434us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.462671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.462688 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:03:39.462726 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.462729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.462779 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.462883 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.464472 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.465652 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.466936 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.467539 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.467687 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.472261 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.472267 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.472272 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.472305 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.475690 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.476468 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.476474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.476478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.476619 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.477018 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.477024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.477027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.477379 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.477664 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.477916 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.478137 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1316,"thread_start_us":1144,"threads_started":1}
I20260925 11:03:39.484293 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.471727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:43811 (local address 127.0.0.1:36644)
0925 11:03:39.471896 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.471914 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:39.472012 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:39.472348 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:39.472351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:39.472661 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:39.472790 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.472797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.473810 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.473816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.474875 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.474886 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.475539 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.475545 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.475706 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.476252 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:39.476275 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:39.478527 (+ 2252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:39.481676 (+ 3149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.481682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.481686 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.482045 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.482504 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.482508 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.482512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.482707 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.483383 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:39.483388 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:39.483534 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 11:03:39.483683 (+ 149us) client_negotiation.cc:240] Negotiation successful
0925 11:03:39.483943 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:03:39.485172 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.471855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36644 (local address 127.24.159.253:43811)
0925 11:03:39.472039 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.472045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.472084 (+ 39us) server_negotiation.cc:407] Connection header received
0925 11:03:39.472140 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.472143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.472216 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.472312 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.472937 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.473675 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.475002 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.475401 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.475506 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.478694 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.478699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.478702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.478733 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.481562 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.482147 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.482153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.482157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.482362 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.482808 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.482813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.482816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.483160 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.483323 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.484577 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.484789 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 11:03:39.485000 6242 raft_consensus.cc:348] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } }
I20260925 11:03:39.485773 6242 raft_consensus.cc:374] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:39.485922 6242 raft_consensus.cc:729] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e000149c6f93411092117255ae9fc0ea, State: Initialized, Role: FOLLOWER
I20260925 11:03:39.486124 6390 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:03:39.486653 6242 consensus_queue.cc:260] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } }
I20260925 11:03:39.488128 6399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.488562 6399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:03:39.489639 6242 sys_catalog.cc:564] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [sys.catalog]: configured and running, proceeding with master startup.
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334219, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
I20260925 11:03:39.500521 6395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.486751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51124)
0925 11:03:39.486888 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.486904 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:39.487000 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:39.487318 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:39.487321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:39.487630 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:39.487833 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.487844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.489404 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.489409 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.490684 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.490698 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.491725 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.491730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.491876 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.492411 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:39.492432 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:39.494931 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:39.498141 (+ 3210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.498146 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.498150 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.498607 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.498945 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.498949 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.498953 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.499076 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.499651 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:39.499655 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:39.499789 (+ 134us) client_negotiation.cc:769] Sending connection context
0925 11:03:39.499923 (+ 134us) client_negotiation.cc:240] Negotiation successful
0925 11:03:39.500201 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260925 11:03:39.501449 6393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.486943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51124 (local address 127.24.159.252:36097)
0925 11:03:39.487089 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.487094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.487109 (+ 15us) server_negotiation.cc:407] Connection header received
0925 11:03:39.487150 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.487154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.487218 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.487308 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.488007 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.489261 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.490819 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.491446 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.491533 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.495087 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.495092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.495097 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.495127 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.498016 (+ 2889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.498717 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.498722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.498726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.498843 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.499178 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.499182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.499186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.499490 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.499611 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.500944 (+ 1333us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.501155 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260925 11:03:39.501612 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.480418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36658 (local address 127.24.159.253:43811)
0925 11:03:39.482890 (+ 2472us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.482894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.482919 (+ 25us) server_negotiation.cc:407] Connection header received
0925 11:03:39.482955 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.482958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.483008 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.483095 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.484252 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.485326 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.487054 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.487651 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.488337 (+ 686us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.493420 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.493428 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.493434 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.493476 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.497577 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.498569 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.498575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.498579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.498740 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.499189 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.499194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.499198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.499557 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.500805 (+ 1248us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.500934 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.501186 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2337,"thread_start_us":2196,"threads_started":1}
I20260925 11:03:39.506765 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:03:39.507018 6390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a: Bootstrap starting.
I20260925 11:03:39.510951 6390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:39.512352 6390 log.cc:824] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:39.515861 6390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a: No bootstrap required, opened a new log
W20260925 11:03:39.526079 6415 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:03:39.526296 6415 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:03:39.526507 6390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } }
I20260925 11:03:39.527026 6390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:39.527184 6390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e5967a8b4a7405ba61cc3d1013eb11a, State: Initialized, Role: FOLLOWER
I20260925 11:03:39.527840 6390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } }
I20260925 11:03:39.524946 6175 master_runner.cc:186] Error getting master registration for 127.24.159.253:43811: OK, instance_id { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" instance_seqno: 1790334218951916 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:03:39.536015 6418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.536401 6418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:03:39.536602 6175 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 11:03:39.538291 6390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:03:39.555166 6396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.520623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52272 (local address 127.24.159.254:35963)
0925 11:03:39.520818 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.520824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.530564 (+ 9740us) server_negotiation.cc:407] Connection header received
0925 11:03:39.530753 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.530758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.530880 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.530992 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.540392 (+ 9400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.541477 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.543831 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.544527 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.544623 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.547928 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.547935 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.547940 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.547978 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.551669 (+ 3691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.552464 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.552469 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.552473 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.552608 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.553019 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.553024 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.553027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.553382 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.553624 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.554588 (+ 964us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.554800 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 11:03:39.561658 6393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.530910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51128 (local address 127.24.159.252:36097)
0925 11:03:39.531082 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.531087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.539984 (+ 8897us) server_negotiation.cc:407] Connection header received
0925 11:03:39.540141 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.540146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.540220 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.540306 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.545444 (+ 5138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.546525 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.548484 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.549143 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.549239 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.552239 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.552245 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.552250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.552288 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.555896 (+ 3608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.556776 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.556782 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.556786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.556962 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.557380 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.557384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.557388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.557730 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.557959 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.561044 (+ 3085us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.561323 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 11:03:39.563925 6398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.530260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.24.159.253:43811)
0925 11:03:39.530581 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.530587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.537268 (+ 6681us) server_negotiation.cc:407] Connection header received
0925 11:03:39.537551 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.537557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.537641 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.537715 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.540070 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.541178 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.543509 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.544168 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.544262 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.557398 (+ 13136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.557405 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.557410 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.557466 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.560439 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.561243 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.561249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.561253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.561391 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.561825 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.561829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.561832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.562149 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.562455 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.563395 (+ 940us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.563629 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
W20260925 11:03:39.569917 6431 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:03:39.570091 6431 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:03:39.572650 6323 master_runner.cc:186] Error getting master registration for 127.24.159.253:43811: OK, instance_id { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" instance_seqno: 1790334218951916 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:03:39.580601 6323 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260925 11:03:39.662238 6344 tablet.cc:2369] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:03:39.671088 6316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } attempt: 5
Sep 25 11:03:39 dist-test-slave-trms krb5kdc[6162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334218, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
I20260925 11:03:39.686162 6418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:03:39.686486 6418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } }
I20260925 11:03:39.688112 6418 leader_election.cc:288] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e000149c6f93411092117255ae9fc0ea (127.24.159.254:35963), 6ec0c40d1b134b9397c961358e1b44f8 (127.24.159.253:43811)
I20260925 11:03:39.689318 6319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.671739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:36097 (local address 127.0.0.1:51140)
0925 11:03:39.671997 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:39.672028 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:39.672149 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:39.672499 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:39.672504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:39.673024 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:39.673226 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.673237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.674732 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.674739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.676166 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:39.676181 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.677460 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.677477 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:39.677704 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.679042 (+ 1338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:39.679066 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:39.681862 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:39.686102 (+ 4240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.686108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.686447 (+ 339us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.686919 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.687365 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:39.687370 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:39.687373 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:39.687564 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:39.688301 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:39.688306 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:39.688461 (+ 155us) client_negotiation.cc:769] Sending connection context
0925 11:03:39.688614 (+ 153us) client_negotiation.cc:240] Negotiation successful
0925 11:03:39.688916 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260925 11:03:39.690475 6393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:39.671820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51140 (local address 127.24.159.252:36097)
0925 11:03:39.671999 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:39.672006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:39.672099 (+ 93us) server_negotiation.cc:407] Connection header received
0925 11:03:39.672268 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:39.672275 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:39.672353 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:39.672459 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:39.673376 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.674600 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.676312 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:39.677134 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:39.677256 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:39.682048 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:39.682054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:39.682060 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:39.682099 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:39.685708 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.687073 (+ 1365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.687079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.687082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.687227 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:39.687689 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:39.687694 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:39.687698 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:39.688085 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:39.688242 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:39.689652 (+ 1410us) server_negotiation.cc:299] Negotiation successful
0925 11:03:39.689957 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 11:03:39.690347 6217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e000149c6f93411092117255ae9fc0ea" is_pre_election: true
I20260925 11:03:39.690989 6217 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e5967a8b4a7405ba61cc3d1013eb11a in term 0.
I20260925 11:03:39.691838 6340 leader_election.cc:302] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e5967a8b4a7405ba61cc3d1013eb11a, e000149c6f93411092117255ae9fc0ea; no voters:
I20260925 11:03:39.692490 6418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:03:39.692632 6418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:03:39.692735 6418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:39.695843 6418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } }
I20260925 11:03:39.696236 6316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8: Bootstrap starting.
I20260925 11:03:39.697111 6217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e000149c6f93411092117255ae9fc0ea"
I20260925 11:03:39.697325 6217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:39.697484 6418 leader_election.cc:288] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [CANDIDATE]: Term 1 election: Requested vote from peers e000149c6f93411092117255ae9fc0ea (127.24.159.254:35963), 6ec0c40d1b134b9397c961358e1b44f8 (127.24.159.253:43811)
I20260925 11:03:39.701361 6217 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e5967a8b4a7405ba61cc3d1013eb11a in term 1.
I20260925 11:03:39.701951 6340 leader_election.cc:302] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e5967a8b4a7405ba61cc3d1013eb11a, e000149c6f93411092117255ae9fc0ea; no voters:
I20260925 11:03:39.702523 6418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:03:39.706655 6316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:39.706854 6418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [term 1 LEADER]: Becoming Leader. State: Replica: 7e5967a8b4a7405ba61cc3d1013eb11a, State: Running, Role: LEADER
I20260925 11:03:39.707404 6418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } }
I20260925 11:03:39.710807 6316 log.cc:824] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:39.712257 6433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e5967a8b4a7405ba61cc3d1013eb11a. Latest consensus state: current_term: 1 leader_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.712612 6433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:39.715127 6316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8: No bootstrap required, opened a new log
W20260925 11:03:39.726549 6196 tablet.cc:2369] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:03:39.726308 6316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } }
I20260925 11:03:39.726724 6316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:39.726840 6316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ec0c40d1b134b9397c961358e1b44f8, State: Initialized, Role: FOLLOWER
I20260925 11:03:39.727596 6316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } }
I20260925 11:03:39.729326 6436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:03:39.733542 6436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:03:39.735126 6437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.735560 6437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:03:39.736581 6316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:03:39.736608 6290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ec0c40d1b134b9397c961358e1b44f8"
I20260925 11:03:39.737063 6290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:39.741312 6291 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ec0c40d1b134b9397c961358e1b44f8" is_pre_election: true
I20260925 11:03:39.745949 6290 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e5967a8b4a7405ba61cc3d1013eb11a in term 1.
I20260925 11:03:39.761422 6290 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [term 1 FOLLOWER]: Refusing update from remote peer 7e5967a8b4a7405ba61cc3d1013eb11a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:03:39.762606 6433 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:03:39.773416 6217 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [term 1 FOLLOWER]: Refusing update from remote peer 7e5967a8b4a7405ba61cc3d1013eb11a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:03:39.774397 6433 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:03:39.807308 6399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e5967a8b4a7405ba61cc3d1013eb11a. Latest consensus state: current_term: 1 leader_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.807648 6399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:03:39.815060 6440 mvcc.cc:205] Tried to move back new op lower bound from 7333208964129210368 to 7333208963925893120. Current Snapshot: MvccSnapshot[applied={T|T < 7333208964129210368}]
I20260925 11:03:39.828054 6447 mvcc.cc:205] Tried to move back new op lower bound from 7333208964129210368 to 7333208963925893120. Current Snapshot: MvccSnapshot[applied={T|T < 7333208964129210368}]
I20260925 11:03:39.840992 6418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.841323 6418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:39.843914 6436 catalog_manager.cc:1347] Generated new cluster ID: 84832a0344994f5eab5e85d3411ff78e
I20260925 11:03:39.844013 6436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:03:39.858126 6399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.858420 6399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:03:39.867517 6437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e5967a8b4a7405ba61cc3d1013eb11a. Latest consensus state: current_term: 1 leader_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.867852 6437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:03:39.876436 6436 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:03:39.882855 6418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.883154 6418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:39.884922 6445 mvcc.cc:205] Tried to move back new op lower bound from 7333208964129210368 to 7333208963925893120. Current Snapshot: MvccSnapshot[applied={T|T < 7333208964129210368}]
I20260925 11:03:39.892372 6436 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:03:39.906814 6437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e000149c6f93411092117255ae9fc0ea" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35963 } } peers { permanent_uuid: "6ec0c40d1b134b9397c961358e1b44f8" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 43811 } } peers { permanent_uuid: "7e5967a8b4a7405ba61cc3d1013eb11a" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 36097 } } }
I20260925 11:03:39.907125 6437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:03:39.912323 6436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e5967a8b4a7405ba61cc3d1013eb11a: Generated new TSK 0
I20260925 11:03:39.913935 6436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:03:39.923205 6455 catalog_manager.cc:1259] Loaded cluster ID: 84832a0344994f5eab5e85d3411ff78e
I20260925 11:03:39.923391 6455 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8: loading cluster ID for follower catalog manager: success
I20260925 11:03:39.927829 6455 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8: acquiring CA information for follower catalog manager: success
I20260925 11:03:39.936744 6455 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ec0c40d1b134b9397c961358e1b44f8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 11:03:40.528530 6415 catalog_manager.cc:1259] Loaded cluster ID: 84832a0344994f5eab5e85d3411ff78e
I20260925 11:03:40.528635 6415 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea: loading cluster ID for follower catalog manager: success
I20260925 11:03:40.531150 6415 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea: acquiring CA information for follower catalog manager: success
I20260925 11:03:40.532883 6415 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e000149c6f93411092117255ae9fc0ea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 11:03:40.555778 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 6175
I20260925 11:03:40.557001 6237 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:40.664927 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 6175
I20260925 11:03:40.681892 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 6249
I20260925 11:03:40.682956 6311 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:40.787164 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 6249
I20260925 11:03:40.803004 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 6323
I20260925 11:03:40.804024 6385 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:40.910946 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 6323
I20260925 11:03:40.911001 6466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:40.910240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:35963 (local address 127.0.0.1:45532)
0925 11:03:40.910660 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:40.910805 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:35963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":239,"threads_started":1}
2026-09-25T11:03:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4037 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:03:40 dist-test-slave-trms krb5kdc[6469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:03:40 dist-test-slave-trms krb5kdc[6469](info): set up 2 sockets
Sep 25 11:03:40 dist-test-slave-trms krb5kdc[6469](info): commencing operation
krb5kdc: starting...
W20260925 11:03:41.910718 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.006s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:03:42 dist-test-slave-trms krb5kdc[6469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:03:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:03:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:03:42.135766 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:41057
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:41057
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:42.453248 6482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:42.453547 6482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:42.453685 6482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:42.460922 6482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:03:42.461050 6482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:42.461131 6482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:42.461210 6482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:03:42.461270 6482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:03:42.471285 6482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:41057
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:41057
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:42.472638 6482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:42.473946 6482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:42.485095 6488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:42.485277 6487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:42.485064 6490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:42.485967 6482 server_base.cc:1034] running on GCE node
I20260925 11:03:42.486733 6482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:42.488090 6482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:42.489289 6482 hybrid_clock.cc:648] HybridClock initialized: now 1790334222489235 us; error 46 us; skew 500 ppm
I20260925 11:03:42.489609 6482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:42 dist-test-slave-trms krb5kdc[6469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334222, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:42.492445 6482 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:03:42.493865 6482 webserver.cc:460] Webserver started at http://127.24.159.254:46655/ using document root <none> and password file <none>
I20260925 11:03:42.494556 6482 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:42.494694 6482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:42.494976 6482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:03:42.497824 6482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "4aa034cdc59645a3bdeb3c03041c9d30"
format_stamp: "Formatted at 2026-09-25 11:03:42 on dist-test-slave-trms"
server_key: "23f2c179b6aca375de178ff3716fbf5d"
server_key_iv: "9407fd98d3ae6a6ff9c5e9140549f33e"
server_key_version: "encryptionkey@0"
I20260925 11:03:42.498561 6482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "4aa034cdc59645a3bdeb3c03041c9d30"
format_stamp: "Formatted at 2026-09-25 11:03:42 on dist-test-slave-trms"
server_key: "23f2c179b6aca375de178ff3716fbf5d"
server_key_iv: "9407fd98d3ae6a6ff9c5e9140549f33e"
server_key_version: "encryptionkey@0"
I20260925 11:03:42.503592 6482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 11:03:42.506865 6497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:42.508205 6482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:03:42.508363 6482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "4aa034cdc59645a3bdeb3c03041c9d30"
format_stamp: "Formatted at 2026-09-25 11:03:42 on dist-test-slave-trms"
server_key: "23f2c179b6aca375de178ff3716fbf5d"
server_key_iv: "9407fd98d3ae6a6ff9c5e9140549f33e"
server_key_version: "encryptionkey@0"
I20260925 11:03:42.508538 6482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:42.537186 6482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:42.540995 6482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:42.541258 6482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:42.557721 6482 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:41057
I20260925 11:03:42.557711 6548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:41057 every 8 connection(s)
I20260925 11:03:42.559118 6482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:03:42.562652 6549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:42.566591 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 6482
I20260925 11:03:42.566803 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:03:42.567587 25215 external_mini_cluster.cc:1428] Setting key 09d8eb539c86895ff43da5d95b459577
I20260925 11:03:42.573629 6549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30: Bootstrap starting.
I20260925 11:03:42.576666 6549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:42.577626 6549 log.cc:824] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:42.581004 6549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30: No bootstrap required, opened a new log
Sep 25 11:03:42 dist-test-slave-trms krb5kdc[6469](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:42.589697 6549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa034cdc59645a3bdeb3c03041c9d30" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41057 } }
I20260925 11:03:42.590056 6549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:42.590160 6549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aa034cdc59645a3bdeb3c03041c9d30, State: Initialized, Role: FOLLOWER
I20260925 11:03:42.590296 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:42.569452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54836 (local address 127.24.159.254:41057)
0925 11:03:42.570155 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:42.570166 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:42.570231 (+ 65us) server_negotiation.cc:407] Connection header received
0925 11:03:42.572415 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:42.572439 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:42.572830 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:42.573195 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:42.574682 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:42.575988 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:42.577361 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:42.578104 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:42.578348 (+ 244us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:42.582978 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:42.582986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:42.583002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:42.583044 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:42.586078 (+ 3034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:42.586926 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:42.586932 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:42.586938 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:42.587116 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:42.587520 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:42.587523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:42.587525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:42.588097 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:42.588341 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:42.588760 (+ 419us) server_negotiation.cc:299] Negotiation successful
0925 11:03:42.589160 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":301,"threads_started":1}
I20260925 11:03:42.590830 6549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa034cdc59645a3bdeb3c03041c9d30" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41057 } }
I20260925 11:03:42.591132 6549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:03:42.591275 6549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:03:42.591449 6549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:42.593978 6549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa034cdc59645a3bdeb3c03041c9d30" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41057 } }
I20260925 11:03:42.594257 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:03:42.594538 6549 leader_election.cc:302] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4aa034cdc59645a3bdeb3c03041c9d30; no voters:
I20260925 11:03:42.595106 6549 leader_election.cc:288] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:03:42.595219 6554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:03:42.597325 6549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:03:42.601804 6554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [term 1 LEADER]: Becoming Leader. State: Replica: 4aa034cdc59645a3bdeb3c03041c9d30, State: Running, Role: LEADER
I20260925 11:03:42.602368 6554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa034cdc59645a3bdeb3c03041c9d30" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41057 } }
I20260925 11:03:42.604710 6555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4aa034cdc59645a3bdeb3c03041c9d30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa034cdc59645a3bdeb3c03041c9d30" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41057 } } }
I20260925 11:03:42.604851 6556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4aa034cdc59645a3bdeb3c03041c9d30. Latest consensus state: current_term: 1 leader_uuid: "4aa034cdc59645a3bdeb3c03041c9d30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4aa034cdc59645a3bdeb3c03041c9d30" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41057 } } }
I20260925 11:03:42.605118 6568 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 11:03:42.605165 6555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:42.605163 6556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:42.606673 6569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:03:42.610592 6569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:03:42.618851 6569 catalog_manager.cc:1347] Generated new cluster ID: 04b4b0f71835442e8121337997cf5a14
I20260925 11:03:42.618944 6569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:03:42.639369 6569 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:03:42.640664 6569 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:03:42.652563 6569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4aa034cdc59645a3bdeb3c03041c9d30: Generated new TSK 0
I20260925 11:03:42.653405 6569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:03:42.706727 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 6482
I20260925 11:03:42.707970 6544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:42.803401 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 6482
2026-09-25T11:03:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1889 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:03:42 dist-test-slave-trms krb5kdc[6577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:03:42 dist-test-slave-trms krb5kdc[6577](info): set up 2 sockets
Sep 25 11:03:42 dist-test-slave-trms krb5kdc[6577](info): commencing operation
krb5kdc: starting...
W20260925 11:03:43.814634 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:03:43 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:03:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:03:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:03:44.027796 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44035
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:44035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:44.338070 6590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:44.338399 6590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:44.338519 6590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:44.345613 6590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:03:44.345726 6590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:44.345800 6590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:44.345880 6590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:03:44.345937 6590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:03:44.355501 6590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:44035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:44.356766 6590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:44.358037 6590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:44.369251 6595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:44.369207 6598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:44.369275 6596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:44.370092 6590 server_base.cc:1034] running on GCE node
I20260925 11:03:44.370885 6590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:44.372282 6590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:44.373492 6590 hybrid_clock.cc:648] HybridClock initialized: now 1790334224373432 us; error 49 us; skew 500 ppm
I20260925 11:03:44.373823 6590 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:44 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334224, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:44.376840 6590 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:03:44.378410 6590 webserver.cc:460] Webserver started at http://127.24.159.254:42961/ using document root <none> and password file <none>
I20260925 11:03:44.379086 6590 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:44.379202 6590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:44.379478 6590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:03:44.382460 6590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "d8602802f45db4edb2a6b8e51812b0ff"
server_key_iv: "842d3bc1ad75141e8694a72fcefe9148"
server_key_version: "encryptionkey@0"
I20260925 11:03:44.383204 6590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "d8602802f45db4edb2a6b8e51812b0ff"
server_key_iv: "842d3bc1ad75141e8694a72fcefe9148"
server_key_version: "encryptionkey@0"
I20260925 11:03:44.388324 6590 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:03:44.391770 6605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:44.393203 6590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:03:44.393383 6590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "d8602802f45db4edb2a6b8e51812b0ff"
server_key_iv: "842d3bc1ad75141e8694a72fcefe9148"
server_key_version: "encryptionkey@0"
I20260925 11:03:44.393572 6590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:44.413261 6590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:44.417538 6590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:44.417804 6590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:44.434793 6590 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:44035
I20260925 11:03:44.434815 6656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:44035 every 8 connection(s)
I20260925 11:03:44.436177 6590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:03:44.439545 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 6590
I20260925 11:03:44.439718 6657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:44.439749 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:03:44.440356 25215 external_mini_cluster.cc:1428] Setting key f24a0228de779ec7988c92cf32389ad5
I20260925 11:03:44.453209 6657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Bootstrap starting.
Sep 25 11:03:44 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:44.456979 6657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:44.458397 6657 log.cc:824] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:44.461710 6657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: No bootstrap required, opened a new log
I20260925 11:03:44.464092 6660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:44.442221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36396 (local address 127.24.159.254:44035)
0925 11:03:44.443009 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:44.443021 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:44.443062 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:03:44.445366 (+ 2304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:44.445391 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:44.445823 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:44.446186 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:44.447894 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:44.449212 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:44.451168 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:44.451852 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:44.451983 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:44.456627 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:44.456638 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:44.456658 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:44.456710 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:44.459986 (+ 3276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:44.460840 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:44.460847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:44.460854 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:44.461018 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:44.461412 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:44.461415 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:44.461417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:44.461969 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:44.462272 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:44.462685 (+ 413us) server_negotiation.cc:299] Negotiation successful
0925 11:03:44.463032 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":345,"threads_started":1}
I20260925 11:03:44.469569 6657 raft_consensus.cc:348] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:44.469936 6657 raft_consensus.cc:374] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:44.470028 6657 raft_consensus.cc:729] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b588f0d3247d4b0f9e91bbb22ecc1cb1, State: Initialized, Role: FOLLOWER
I20260925 11:03:44.470602 6657 consensus_queue.cc:260] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:44.470817 6657 raft_consensus.cc:388] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:03:44.470943 6657 raft_consensus.cc:482] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:03:44.471093 6657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:44.473232 6657 raft_consensus.cc:504] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:44.473734 6657 leader_election.cc:302] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b588f0d3247d4b0f9e91bbb22ecc1cb1; no voters:
I20260925 11:03:44.474354 6657 leader_election.cc:288] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:03:44.474460 6662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:03:44.475114 6662 raft_consensus.cc:686] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 1 LEADER]: Becoming Leader. State: Replica: b588f0d3247d4b0f9e91bbb22ecc1cb1, State: Running, Role: LEADER
I20260925 11:03:44.475630 6662 consensus_queue.cc:237] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:44.476387 6657 sys_catalog.cc:564] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:03:44.478241 6663 sys_catalog.cc:455] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } } }
I20260925 11:03:44.478621 6663 sys_catalog.cc:458] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:44.479454 6667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:03:44.479295 6664 sys_catalog.cc:455] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b588f0d3247d4b0f9e91bbb22ecc1cb1. Latest consensus state: current_term: 1 leader_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } } }
I20260925 11:03:44.480502 6664 sys_catalog.cc:458] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:44.485477 6667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:03:44.495002 6667 catalog_manager.cc:1347] Generated new cluster ID: 250e0a8403004d888da948d43089d2b1
I20260925 11:03:44.495114 6667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:03:44.513300 6667 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:03:44.514850 6667 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:03:44.527329 6667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Generated new TSK 0
I20260925 11:03:44.528530 6667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:03:44.588960 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44035
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:44.905562 6683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:44.905823 6683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:44.905962 6683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:44.912969 6683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:44.913081 6683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:44.913230 6683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:03:44.922788 6683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:44.924015 6683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:44.925324 6683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:44.937693 6688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:44.937747 6691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:44.937721 6689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:44.938486 6683 server_base.cc:1034] running on GCE node
I20260925 11:03:44.939023 6683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:44.939862 6683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:44.941071 6683 hybrid_clock.cc:648] HybridClock initialized: now 1790334224941017 us; error 45 us; skew 500 ppm
I20260925 11:03:44.941370 6683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:44 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334224, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:44.948405 6683 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:03:44.949786 6683 webserver.cc:460] Webserver started at http://127.24.159.193:39551/ using document root <none> and password file <none>
I20260925 11:03:44.950484 6683 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:44.950596 6683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:44.950870 6683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:03:44.953661 6683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "818c806154164b67b3df387895644ffb"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "12490b366fc755fc970ef608c094a56d"
server_key_iv: "be196bf8c1e1dec1760a14057428c5ea"
server_key_version: "encryptionkey@0"
I20260925 11:03:44.954450 6683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "818c806154164b67b3df387895644ffb"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "12490b366fc755fc970ef608c094a56d"
server_key_iv: "be196bf8c1e1dec1760a14057428c5ea"
server_key_version: "encryptionkey@0"
I20260925 11:03:44.959355 6683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:03:44.962678 6698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:44.963953 6683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260925 11:03:44.964082 6683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "818c806154164b67b3df387895644ffb"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "12490b366fc755fc970ef608c094a56d"
server_key_iv: "be196bf8c1e1dec1760a14057428c5ea"
server_key_version: "encryptionkey@0"
I20260925 11:03:44.964255 6683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:44.983626 6683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:44.987432 6683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:44.987663 6683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:44.988533 6683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:44.989787 6683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:44.989856 6683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:44.989946 6683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:44.989986 6683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:45.018422 6683 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:33653
I20260925 11:03:45.018457 6810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:33653 every 8 connection(s)
I20260925 11:03:45.019613 6683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:03:45.029747 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 6683
I20260925 11:03:45.029937 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:03:45.030653 25215 external_mini_cluster.cc:1428] Setting key 3863211c45ed7fd6bd24dc22eabe8f47
Sep 25 11:03:45 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334224, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:45.038311 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:45.021990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:39065 (local address 127.24.159.254:44035)
0925 11:03:45.022435 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:45.022440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:45.023382 (+ 942us) server_negotiation.cc:407] Connection header received
0925 11:03:45.025028 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:45.025033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:45.025099 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:45.025194 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:45.026658 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:45.027471 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:45.028931 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:45.029328 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:45.029447 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:45.032759 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:45.032764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:45.032767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:45.032799 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:45.035249 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:45.036057 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:45.036061 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:45.036065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:45.036198 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:45.036604 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:45.036607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:45.036610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:45.036890 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:45.037016 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:45.037771 (+ 755us) server_negotiation.cc:299] Negotiation successful
0925 11:03:45.037965 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":237,"threads_started":1}
I20260925 11:03:45.039505 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:45.022578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:39065)
0925 11:03:45.023229 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:45.023269 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:45.024804 (+ 1535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:45.025372 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:45.025379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:45.025865 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:45.026501 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:45.026520 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:45.027624 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:45.027627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:45.028818 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:45.028830 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:45.029499 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:45.029503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:45.029693 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:45.030342 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:45.030367 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:45.032556 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:45.035382 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:45.035391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:45.035408 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:45.035952 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:45.036289 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:45.036293 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:45.036297 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:45.036514 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:45.037000 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:45.037008 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:45.037417 (+ 409us) client_negotiation.cc:769] Sending connection context
0925 11:03:45.037684 (+ 267us) client_negotiation.cc:240] Negotiation successful
0925 11:03:45.038227 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":238,"threads_started":1}
I20260925 11:03:45.041492 6811 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44035
I20260925 11:03:45.041772 6811 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:45.042497 6811 heartbeater.cc:511] Master 127.24.159.254:44035 requested a full tablet report, sending...
I20260925 11:03:45.044488 6622 ts_manager.cc:194] Registered new tserver with Master: 818c806154164b67b3df387895644ffb (127.24.159.193:33653)
I20260925 11:03:45.045959 6622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:39065
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:03:45.077646 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44035
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:45.391646 6817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:45.391942 6817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:45.392078 6817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:45.399523 6817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:45.399636 6817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:45.399783 6817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:03:45.409488 6817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:45.410790 6817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:45.412160 6817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:45.424654 6823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:45.424678 6822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:45.424933 6825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:45.425357 6817 server_base.cc:1034] running on GCE node
I20260925 11:03:45.425979 6817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:45.426973 6817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:45.428185 6817 hybrid_clock.cc:648] HybridClock initialized: now 1790334225428143 us; error 47 us; skew 500 ppm
I20260925 11:03:45.428474 6817 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:45 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334225, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:45.431521 6817 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:03:45.432891 6817 webserver.cc:460] Webserver started at http://127.24.159.194:42381/ using document root <none> and password file <none>
I20260925 11:03:45.433545 6817 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:45.433648 6817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:45.433921 6817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:03:45.436810 6817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "736b126b26374837a9e423b9e9bb69e4"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "7229be0c333e1adfa0fc4ee166b60aa9"
server_key_iv: "1a982e8f22fa40a7887d6e7d3979f512"
server_key_version: "encryptionkey@0"
I20260925 11:03:45.437556 6817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "736b126b26374837a9e423b9e9bb69e4"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "7229be0c333e1adfa0fc4ee166b60aa9"
server_key_iv: "1a982e8f22fa40a7887d6e7d3979f512"
server_key_version: "encryptionkey@0"
I20260925 11:03:45.442588 6817 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 11:03:45.446012 6832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:45.447458 6817 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:03:45.447603 6817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "736b126b26374837a9e423b9e9bb69e4"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "7229be0c333e1adfa0fc4ee166b60aa9"
server_key_iv: "1a982e8f22fa40a7887d6e7d3979f512"
server_key_version: "encryptionkey@0"
I20260925 11:03:45.447774 6817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:45.460618 6817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:45.464690 6817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:45.464946 6817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:45.465822 6817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:45.467255 6817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:45.467332 6817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:45.467445 6817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:45.467502 6817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:45.496788 6817 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:39897
I20260925 11:03:45.496855 6944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:39897 every 8 connection(s)
I20260925 11:03:45.498251 6817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:03:45.508533 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 6817
I20260925 11:03:45.508708 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:03:45.509253 25215 external_mini_cluster.cc:1428] Setting key 58039426191430f58ad664cb4c9c2083
Sep 25 11:03:45 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334225, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:45.518059 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:45.501166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:46773 (local address 127.24.159.254:44035)
0925 11:03:45.501351 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:45.501355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:45.502561 (+ 1206us) server_negotiation.cc:407] Connection header received
0925 11:03:45.504306 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:45.504310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:45.504382 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:45.504466 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:45.505894 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:45.506685 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:45.508248 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:45.508619 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:45.508738 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:45.512639 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:45.512644 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:45.512647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:45.512679 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:45.515145 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:45.515969 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:45.515972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:45.515974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:45.516101 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:45.516509 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:45.516512 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:45.516514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:45.516767 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:45.516875 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:45.517654 (+ 779us) server_negotiation.cc:299] Negotiation successful
0925 11:03:45.517827 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 11:03:45.519337 6947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:45.501676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:46773)
0925 11:03:45.502402 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:45.502449 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:45.504063 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:45.504655 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:45.504664 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:45.505199 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:45.505759 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:45.505775 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:45.506843 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:45.506848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:45.508124 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:45.508136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:45.508841 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:45.508850 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:45.509166 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:45.509838 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:45.509860 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:45.512414 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:45.515291 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:45.515300 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:45.515319 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:45.515863 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:45.516191 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:45.516195 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:45.516199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:45.516424 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:45.516874 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:45.516882 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:45.517313 (+ 431us) client_negotiation.cc:769] Sending connection context
0925 11:03:45.517598 (+ 285us) client_negotiation.cc:240] Negotiation successful
0925 11:03:45.518117 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":273,"threads_started":1}
I20260925 11:03:45.520648 6945 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44035
I20260925 11:03:45.520968 6945 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:45.521708 6945 heartbeater.cc:511] Master 127.24.159.254:44035 requested a full tablet report, sending...
I20260925 11:03:45.523147 6622 ts_manager.cc:194] Registered new tserver with Master: 736b126b26374837a9e423b9e9bb69e4 (127.24.159.194:39897)
I20260925 11:03:45.523919 6622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:46773
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:03:45.557747 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44035
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:45.872877 6950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:45.873195 6950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:45.873338 6950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:45.880609 6950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:45.880731 6950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:45.880880 6950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:03:45.894548 6950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:45.896209 6950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:45.897665 6950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:45.911820 6958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:45.911850 6955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:45.912004 6956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:45.912755 6950 server_base.cc:1034] running on GCE node
I20260925 11:03:45.913450 6950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:45.914579 6950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:45.915846 6950 hybrid_clock.cc:648] HybridClock initialized: now 1790334225915771 us; error 54 us; skew 500 ppm
I20260925 11:03:45.916245 6950 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:45 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334225, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:45.920195 6950 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:03:45.921803 6950 webserver.cc:460] Webserver started at http://127.24.159.195:33671/ using document root <none> and password file <none>
I20260925 11:03:45.922683 6950 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:45.922809 6950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:45.923102 6950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:03:45.925957 6950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "4c01a5b49c77401ba10b36c5666591f6"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "b352d226ab2fbdd49a036a9f8817dc1d"
server_key_iv: "daac3039ba71c18b94284f893c506ea6"
server_key_version: "encryptionkey@0"
I20260925 11:03:45.926868 6950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "4c01a5b49c77401ba10b36c5666591f6"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "b352d226ab2fbdd49a036a9f8817dc1d"
server_key_iv: "daac3039ba71c18b94284f893c506ea6"
server_key_version: "encryptionkey@0"
I20260925 11:03:45.932191 6950 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:03:45.935595 6965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:45.936934 6950 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260925 11:03:45.937098 6950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "4c01a5b49c77401ba10b36c5666591f6"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "b352d226ab2fbdd49a036a9f8817dc1d"
server_key_iv: "daac3039ba71c18b94284f893c506ea6"
server_key_version: "encryptionkey@0"
I20260925 11:03:45.937306 6950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:45.973735 6950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:45.977785 6950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:45.978046 6950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:45.979038 6950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:45.980348 6950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:45.980434 6950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:45.980558 6950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:45.980609 6950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:46.010032 6950 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:37427
I20260925 11:03:46.010074 7077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:37427 every 8 connection(s)
I20260925 11:03:46.011356 6950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:03:46.018642 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 6950
I20260925 11:03:46.018844 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:03:46.019377 25215 external_mini_cluster.cc:1428] Setting key 9978f80c810597feb02940b5a23df637
Sep 25 11:03:46 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334225, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:46.032733 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.013852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:40461 (local address 127.24.159.254:44035)
0925 11:03:46.014031 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.014036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.015268 (+ 1232us) server_negotiation.cc:407] Connection header received
0925 11:03:46.017101 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.017106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.017189 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.017278 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:46.018956 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.019694 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.022178 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.022821 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.022950 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.026134 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:46.026139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:46.026142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:46.026174 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:46.028861 (+ 2687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.029755 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.029761 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.029765 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.029908 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.030517 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.030523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.030526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.030877 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:46.031034 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.032014 (+ 980us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.032351 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:03:46.033739 7080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.014406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:40461)
0925 11:03:46.015104 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:46.015154 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:46.016811 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:46.017532 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:46.017541 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:46.018137 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:46.018776 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.018805 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.020205 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.020210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.022030 (+ 1820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.022044 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.023034 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.023039 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.023232 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.023842 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:46.023864 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:46.025923 (+ 2059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:46.029020 (+ 3097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:46.029030 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:46.029047 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:46.029651 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:46.030057 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:46.030062 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:46.030066 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:46.030366 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:46.031022 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:46.031030 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:46.031498 (+ 468us) client_negotiation.cc:769] Sending connection context
0925 11:03:46.031824 (+ 326us) client_negotiation.cc:240] Negotiation successful
0925 11:03:46.032354 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":278,"threads_started":1}
I20260925 11:03:46.035576 7078 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44035
I20260925 11:03:46.035844 7078 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:46.036558 7078 heartbeater.cc:511] Master 127.24.159.254:44035 requested a full tablet report, sending...
I20260925 11:03:46.037868 6622 ts_manager.cc:194] Registered new tserver with Master: 4c01a5b49c77401ba10b36c5666591f6 (127.24.159.195:37427)
I20260925 11:03:46.038702 6622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:40461
I20260925 11:03:46.047340 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:03:46.048748 6811 heartbeater.cc:503] Master 127.24.159.254:44035 was elected leader, sending a full tablet report...
I20260925 11:03:46.063243 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.051936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36410 (local address 127.24.159.254:44035)
0925 11:03:46.052172 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.052176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.052506 (+ 330us) server_negotiation.cc:407] Connection header received
0925 11:03:46.052643 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.052647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.052706 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.052776 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:46.054069 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.054835 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.056126 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.056545 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.056640 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.058840 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:46.058845 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:46.058848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:46.058880 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:46.061106 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.061717 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.061720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.061723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.061870 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.062139 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.062142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.062144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.062464 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:46.062637 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.062864 (+ 227us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.063032 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260925 11:03:46.066985 6622 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:03:46.073745 6622 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:03:46.105046 7094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.086008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39897 (local address 127.0.0.1:45028)
0925 11:03:46.097228 (+ 11220us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:46.097248 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:46.097388 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:46.098481 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:46.098485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:46.098511 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:46.098976 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.098990 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.100698 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.100704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.102467 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.102479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.104103 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.104109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.104300 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.104352 (+ 52us) client_negotiation.cc:769] Sending connection context
0925 11:03:46.104512 (+ 160us) client_negotiation.cc:240] Negotiation successful
0925 11:03:46.104708 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11074,"spinlock_wait_cycles":25344,"thread_start_us":9266,"threads_started":1}
I20260925 11:03:46.105710 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.086562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49154 (local address 127.24.159.195:37427)
0925 11:03:46.095094 (+ 8532us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.095103 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.095912 (+ 809us) server_negotiation.cc:407] Connection header received
0925 11:03:46.096199 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.096207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.096496 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.096771 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:46.097930 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.099773 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.102575 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.104020 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.104204 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.104392 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.105159 (+ 767us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.105336 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8317,"thread_start_us":7811,"threads_started":1}
I20260925 11:03:46.106060 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.086933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.24.159.194:39897)
0925 11:03:46.097878 (+ 10945us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.097887 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.097914 (+ 27us) server_negotiation.cc:407] Connection header received
0925 11:03:46.098002 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.098008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.098310 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.098489 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:46.099175 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.100576 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.102612 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.103552 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.103702 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.103885 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.105445 (+ 1560us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.105616 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10814,"thread_start_us":10677,"threads_started":1}
I20260925 11:03:46.105465 7090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.085973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:37427 (local address 127.0.0.1:49154)
0925 11:03:46.095745 (+ 9772us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:46.095784 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:46.095978 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:46.097117 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:46.097125 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:46.097161 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:46.097703 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.097718 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.099944 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.099952 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.102367 (+ 2415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.102382 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.104543 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.104549 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.104753 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.104786 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 11:03:46.104880 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 11:03:46.105008 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9439,"mutex_wait_us":5,"spinlock_wait_cycles":9344,"thread_start_us":13878,"threads_started":1}
I20260925 11:03:46.107791 7091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.087987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:33653 (local address 127.0.0.1:59174)
0925 11:03:46.100150 (+ 12163us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:46.100169 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:46.100284 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:46.100875 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:46.100879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:46.100908 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:46.101382 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.101399 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.103246 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.103251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.105246 (+ 1995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.105259 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.107073 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.107079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.107233 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.107265 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 11:03:46.107355 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 11:03:46.107490 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12015,"thread_start_us":8958,"threads_started":1}
I20260925 11:03:46.107987 7095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.088597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59174 (local address 127.24.159.193:33653)
0925 11:03:46.096783 (+ 8186us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.096793 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.100286 (+ 3493us) server_negotiation.cc:407] Connection header received
0925 11:03:46.100418 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.100426 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.100671 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.100873 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:46.101568 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.103070 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.105426 (+ 2356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.106865 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.107059 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.107227 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.107410 (+ 183us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.107561 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8001,"thread_start_us":7843,"threads_started":1}
I20260925 11:03:46.110513 6880 tablet_service.cc:1467] Processing CreateTablet for tablet 2aef4c57ecc34795a7dc98172f9c77a5 (DEFAULT_TABLE table=test-table [id=b7f11c6078184d1cbc7bb2ca8ab8c0dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:03:46.110512 7013 tablet_service.cc:1467] Processing CreateTablet for tablet 2aef4c57ecc34795a7dc98172f9c77a5 (DEFAULT_TABLE table=test-table [id=b7f11c6078184d1cbc7bb2ca8ab8c0dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:03:46.110873 6746 tablet_service.cc:1467] Processing CreateTablet for tablet 2aef4c57ecc34795a7dc98172f9c77a5 (DEFAULT_TABLE table=test-table [id=b7f11c6078184d1cbc7bb2ca8ab8c0dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:03:46.111989 6746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aef4c57ecc34795a7dc98172f9c77a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:46.111979 7013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aef4c57ecc34795a7dc98172f9c77a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:46.112004 6880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aef4c57ecc34795a7dc98172f9c77a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:46.122689 7096 tablet_bootstrap.cc:492] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: Bootstrap starting.
I20260925 11:03:46.125593 7096 tablet_bootstrap.cc:654] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:46.126775 7096 log.cc:824] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:46.126825 7097 tablet_bootstrap.cc:492] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: Bootstrap starting.
I20260925 11:03:46.128684 7098 tablet_bootstrap.cc:492] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: Bootstrap starting.
I20260925 11:03:46.130371 7097 tablet_bootstrap.cc:654] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:46.130718 7096 tablet_bootstrap.cc:492] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: No bootstrap required, opened a new log
I20260925 11:03:46.131036 7096 ts_tablet_manager.cc:1397] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.002s
I20260925 11:03:46.131717 7097 log.cc:824] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:46.132076 7098 tablet_bootstrap.cc:654] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:46.133303 7098 log.cc:824] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:46.136062 7098 tablet_bootstrap.cc:492] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: No bootstrap required, opened a new log
I20260925 11:03:46.136296 7098 ts_tablet_manager.cc:1397] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260925 11:03:46.137614 7097 tablet_bootstrap.cc:492] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: No bootstrap required, opened a new log
I20260925 11:03:46.137866 7097 ts_tablet_manager.cc:1397] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260925 11:03:46.139420 7096 raft_consensus.cc:348] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:46.139851 7096 raft_consensus.cc:374] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:46.139989 7096 raft_consensus.cc:729] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818c806154164b67b3df387895644ffb, State: Initialized, Role: FOLLOWER
I20260925 11:03:46.140707 7096 consensus_queue.cc:260] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:46.143563 7096 ts_tablet_manager.cc:1428] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: Time spent starting tablet: real 0.012s user 0.009s sys 0.002s
I20260925 11:03:46.144351 7098 raft_consensus.cc:348] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:46.144706 7098 raft_consensus.cc:374] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:46.144798 7098 raft_consensus.cc:729] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c01a5b49c77401ba10b36c5666591f6, State: Initialized, Role: FOLLOWER
I20260925 11:03:46.145328 7098 consensus_queue.cc:260] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:46.146330 7078 heartbeater.cc:503] Master 127.24.159.254:44035 was elected leader, sending a full tablet report...
I20260925 11:03:46.147786 7097 raft_consensus.cc:348] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:46.148341 7098 ts_tablet_manager.cc:1428] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: Time spent starting tablet: real 0.012s user 0.006s sys 0.005s
I20260925 11:03:46.148411 7097 raft_consensus.cc:374] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:46.148582 7097 raft_consensus.cc:729] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736b126b26374837a9e423b9e9bb69e4, State: Initialized, Role: FOLLOWER
I20260925 11:03:46.149286 7097 consensus_queue.cc:260] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:46.150406 6945 heartbeater.cc:503] Master 127.24.159.254:44035 was elected leader, sending a full tablet report...
I20260925 11:03:46.150931 7097 ts_tablet_manager.cc:1428] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260925 11:03:46.250574 6946 tablet.cc:2369] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:03:46.262986 7079 tablet.cc:2369] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:03:46.271733 6812 tablet.cc:2369] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:03:46.343163 7102 raft_consensus.cc:482] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:03:46.343441 7102 raft_consensus.cc:504] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:46.345037 7102 leader_election.cc:288] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 736b126b26374837a9e423b9e9bb69e4 (127.24.159.194:39897), 4c01a5b49c77401ba10b36c5666591f6 (127.24.159.195:37427)
I20260925 11:03:46.351393 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.345458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:32985 (local address 127.24.159.194:39897)
0925 11:03:46.345719 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.345726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.346161 (+ 435us) server_negotiation.cc:407] Connection header received
0925 11:03:46.346404 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.346408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.346475 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.346557 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:46.347056 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.347766 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.349650 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.350512 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.350654 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.350728 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.351026 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.351145 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260925 11:03:46.351495 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.345493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:42877 (local address 127.24.159.195:37427)
0925 11:03:46.345890 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.345896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.346476 (+ 580us) server_negotiation.cc:407] Connection header received
0925 11:03:46.346640 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.346644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.346701 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.346773 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:46.347299 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.348086 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.349807 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.350652 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.350787 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.350856 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.351132 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.351249 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260925 11:03:46.351547 7106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.345532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:37427 (local address 127.24.159.193:42877)
0925 11:03:46.346379 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:46.346392 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:46.346499 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:46.346805 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:46.346813 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:46.346836 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:46.347161 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.347169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.348208 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.348213 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.349637 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.349649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.350846 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.350852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.350982 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.351003 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 11:03:46.351081 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 11:03:46.351193 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":527,"threads_started":1}
I20260925 11:03:46.351588 7105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.345215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39897 (local address 127.24.159.193:32985)
0925 11:03:46.346054 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:46.346074 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:46.346265 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:46.346572 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:46.346576 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:46.346599 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:46.346924 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.346933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.347893 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.347899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.349489 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:46.349501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.350750 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.350757 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:46.350886 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.350910 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 11:03:46.350990 (+ 80us) client_negotiation.cc:240] Negotiation successful
0925 11:03:46.351113 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":18,"thread_start_us":528,"threads_started":1}
I20260925 11:03:46.352711 7033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef4c57ecc34795a7dc98172f9c77a5" candidate_uuid: "818c806154164b67b3df387895644ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c01a5b49c77401ba10b36c5666591f6" is_pre_election: true
I20260925 11:03:46.352710 6900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef4c57ecc34795a7dc98172f9c77a5" candidate_uuid: "818c806154164b67b3df387895644ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736b126b26374837a9e423b9e9bb69e4" is_pre_election: true
I20260925 11:03:46.353259 7033 raft_consensus.cc:2454] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 818c806154164b67b3df387895644ffb in term 0.
I20260925 11:03:46.353259 6900 raft_consensus.cc:2454] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 818c806154164b67b3df387895644ffb in term 0.
I20260925 11:03:46.353901 6700 leader_election.cc:302] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c01a5b49c77401ba10b36c5666591f6, 818c806154164b67b3df387895644ffb; no voters:
I20260925 11:03:46.354279 7102 raft_consensus.cc:2781] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:03:46.354415 7102 raft_consensus.cc:482] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:03:46.354528 7102 raft_consensus.cc:3037] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:46.356633 7102 raft_consensus.cc:504] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:46.357209 7102 leader_election.cc:288] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [CANDIDATE]: Term 1 election: Requested vote from peers 736b126b26374837a9e423b9e9bb69e4 (127.24.159.194:39897), 4c01a5b49c77401ba10b36c5666591f6 (127.24.159.195:37427)
I20260925 11:03:46.357620 6900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef4c57ecc34795a7dc98172f9c77a5" candidate_uuid: "818c806154164b67b3df387895644ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736b126b26374837a9e423b9e9bb69e4"
I20260925 11:03:46.357679 7033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef4c57ecc34795a7dc98172f9c77a5" candidate_uuid: "818c806154164b67b3df387895644ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c01a5b49c77401ba10b36c5666591f6"
I20260925 11:03:46.357801 6900 raft_consensus.cc:3037] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:46.357885 7033 raft_consensus.cc:3037] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:46.360311 6900 raft_consensus.cc:2454] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 818c806154164b67b3df387895644ffb in term 1.
I20260925 11:03:46.360320 7033 raft_consensus.cc:2454] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 818c806154164b67b3df387895644ffb in term 1.
I20260925 11:03:46.360679 6702 leader_election.cc:302] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 736b126b26374837a9e423b9e9bb69e4, 818c806154164b67b3df387895644ffb; no voters:
I20260925 11:03:46.360970 7102 raft_consensus.cc:2781] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:03:46.361413 7102 raft_consensus.cc:686] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 1 LEADER]: Becoming Leader. State: Replica: 818c806154164b67b3df387895644ffb, State: Running, Role: LEADER
I20260925 11:03:46.361914 7102 consensus_queue.cc:237] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:46.366238 6621 catalog_manager.cc:5495] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb reported cstate change: term changed from 0 to 1, leader changed from <none> to 818c806154164b67b3df387895644ffb (127.24.159.193). New cstate: current_term: 1 leader_uuid: "818c806154164b67b3df387895644ffb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } health_report { overall_health: HEALTHY } } }
I20260925 11:03:46.396413 7095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.391205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59180 (local address 127.24.159.193:33653)
0925 11:03:46.391453 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.391459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.391482 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:03:46.391670 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.391675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.391787 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.391985 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:03:46.392384 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.393105 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.394515 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.394909 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.395014 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.395454 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:03:46.395962 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:03:46.396069 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.396106 (+ 37us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.396171 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260925 11:03:46.404793 7033 raft_consensus.cc:1261] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [term 1 FOLLOWER]: Refusing update from remote peer 818c806154164b67b3df387895644ffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:03:46.404793 6900 raft_consensus.cc:1261] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [term 1 FOLLOWER]: Refusing update from remote peer 818c806154164b67b3df387895644ffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:03:46.405740 7107 consensus_queue.cc:1035] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:03:46.406057 7102 consensus_queue.cc:1035] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:03:46.416518 7110 mvcc.cc:205] Tried to move back new op lower bound from 7333208991344500736 to 7333208991182016512. Current Snapshot: MvccSnapshot[applied={T|T < 7333208991344500736}]
I20260925 11:03:46.417827 7111 mvcc.cc:205] Tried to move back new op lower bound from 7333208991344500736 to 7333208991182016512. Current Snapshot: MvccSnapshot[applied={T|T < 7333208991344500736}]
I20260925 11:03:46.423136 7112 mvcc.cc:205] Tried to move back new op lower bound from 7333208991344500736 to 7333208991182016512. Current Snapshot: MvccSnapshot[applied={T|T < 7333208991344500736}]
I20260925 11:03:46.436722 6622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:03:46.437037 6622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:03:46.440902 6622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Sending DeleteTablet for 3 replicas of tablet 2aef4c57ecc34795a7dc98172f9c77a5
I20260925 11:03:46.442031 7013 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aef4c57ecc34795a7dc98172f9c77a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:46 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:49154
I20260925 11:03:46.442031 6880 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aef4c57ecc34795a7dc98172f9c77a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:46 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:45028
I20260925 11:03:46.443324 6746 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aef4c57ecc34795a7dc98172f9c77a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:46 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:59174
I20260925 11:03:46.444120 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 6683
I20260925 11:03:46.461012 7120 tablet_replica.cc:330] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: stopping tablet replica
I20260925 11:03:46.461572 7120 raft_consensus.cc:2229] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:03:46.462167 7120 raft_consensus.cc:2258] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:03:46.463006 7122 tablet_replica.cc:330] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: stopping tablet replica
I20260925 11:03:46.463425 7122 raft_consensus.cc:2229] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:03:46.464156 7122 raft_consensus.cc:2258] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:03:46.467731 7121 tablet_replica.cc:330] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: stopping tablet replica
I20260925 11:03:46.468143 7121 raft_consensus.cc:2229] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:03:46.469100 7120 ts_tablet_manager.cc:1905] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:03:46.470670 7121 raft_consensus.cc:2258] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:03:46.471310 7122 ts_tablet_manager.cc:1905] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:03:46.473021 7121 ts_tablet_manager.cc:1905] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:03:46.475896 7120 ts_tablet_manager.cc:1918] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:03:46.476120 7120 log.cc:1196] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/2aef4c57ecc34795a7dc98172f9c77a5
I20260925 11:03:46.476678 7120 ts_tablet_manager.cc:1939] T 2aef4c57ecc34795a7dc98172f9c77a5 P 4c01a5b49c77401ba10b36c5666591f6: Deleting consensus metadata
I20260925 11:03:46.478684 6607 catalog_manager.cc:4841] TS 4c01a5b49c77401ba10b36c5666591f6 (127.24.159.195:37427): tablet 2aef4c57ecc34795a7dc98172f9c77a5 (table test-table [id=b7f11c6078184d1cbc7bb2ca8ab8c0dc]) successfully deleted
I20260925 11:03:46.480404 7121 ts_tablet_manager.cc:1918] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:03:46.480579 7121 log.cc:1196] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/2aef4c57ecc34795a7dc98172f9c77a5
I20260925 11:03:46.481084 7121 ts_tablet_manager.cc:1939] T 2aef4c57ecc34795a7dc98172f9c77a5 P 736b126b26374837a9e423b9e9bb69e4: Deleting consensus metadata
I20260925 11:03:46.482975 6609 catalog_manager.cc:4841] TS 736b126b26374837a9e423b9e9bb69e4 (127.24.159.194:39897): tablet 2aef4c57ecc34795a7dc98172f9c77a5 (table test-table [id=b7f11c6078184d1cbc7bb2ca8ab8c0dc]) successfully deleted
Sep 25 11:03:46 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:03:46.485035 7122 ts_tablet_manager.cc:1918] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:03:46.485215 7122 log.cc:1196] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/2aef4c57ecc34795a7dc98172f9c77a5
I20260925 11:03:46.485666 7122 ts_tablet_manager.cc:1939] T 2aef4c57ecc34795a7dc98172f9c77a5 P 818c806154164b67b3df387895644ffb: Deleting consensus metadata
I20260925 11:03:46.487408 6606 catalog_manager.cc:4841] TS 818c806154164b67b3df387895644ffb (127.24.159.193:33653): tablet 2aef4c57ecc34795a7dc98172f9c77a5 (table test-table [id=b7f11c6078184d1cbc7bb2ca8ab8c0dc]) successfully deleted
I20260925 11:03:46.491860 7095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.445237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.24.159.193:33653)
0925 11:03:46.445421 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.445428 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.463021 (+ 17593us) server_negotiation.cc:407] Connection header received
0925 11:03:46.463524 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.463530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.463613 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.463704 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:46.465640 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.466744 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.468668 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.469354 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.469499 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.484422 (+ 14923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:46.484444 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:46.484462 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:46.484516 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:46.488881 (+ 4365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.489676 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.489686 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.489690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.489867 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.490246 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.490251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.490254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.490715 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:46.491024 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.491231 (+ 207us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.491487 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 11:03:46.492676 6806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:46.591086 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 6683
I20260925 11:03:46.606442 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 6817
Sep 25 11:03:46 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:03:46.620335 7093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.607251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.24.159.194:39897)
0925 11:03:46.607428 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.607433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.607451 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:03:46.607589 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.607593 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.607658 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.607737 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:46.608843 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.609594 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.610847 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.611222 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.611328 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.615157 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:46.615171 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:46.615189 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:46.615225 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:46.617728 (+ 2503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.618388 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.618398 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.618402 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.618543 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.618840 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.618843 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.618845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.619209 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:46.619415 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.619834 (+ 419us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.620037 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 11:03:46.621008 6940 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:46.708956 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 6817
I20260925 11:03:46.722246 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 6950
Sep 25 11:03:46 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:03:46.736742 7092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:46.722996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.24.159.195:37427)
0925 11:03:46.723172 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:46.723178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:46.723198 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:03:46.723383 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:46.723388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:46.723468 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:46.723553 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:46.724845 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.725548 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.727043 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:46.727435 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:46.727552 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:46.731604 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:46.731621 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:46.731638 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:46.731679 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:46.734039 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.734790 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.734797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.734800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.734945 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:46.735292 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:46.735295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:46.735297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:46.735626 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:46.735894 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:46.736224 (+ 330us) server_negotiation.cc:299] Negotiation successful
0925 11:03:46.736436 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:03:46.737375 7073 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:46.824249 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 6950
I20260925 11:03:46.838621 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 6590
I20260925 11:03:46.839771 6652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:46.948712 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 6590
I20260925 11:03:46.966259 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44035
--webserver_interface=127.24.159.254
--webserver_port=42961
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:44035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:47.285136 7128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:47.285452 7128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:47.285589 7128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:47.292737 7128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:03:47.292847 7128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:47.292927 7128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:47.293001 7128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:03:47.293061 7128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:03:47.302646 7128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:44035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=42961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:47.303936 7128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:47.305198 7128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:47.326089 7134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:47.326056 7135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:47.326234 7137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:47.326978 7128 server_base.cc:1034] running on GCE node
I20260925 11:03:47.327821 7128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:47.329324 7128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:47.330510 7128 hybrid_clock.cc:648] HybridClock initialized: now 1790334227330450 us; error 50 us; skew 500 ppm
I20260925 11:03:47.330891 7128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:47 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334227, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:47.334142 7128 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:03:47.335855 7128 webserver.cc:460] Webserver started at http://127.24.159.254:42961/ using document root <none> and password file <none>
I20260925 11:03:47.336575 7128 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:47.336701 7128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:47.342507 7128 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260925 11:03:47.345242 7144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:47.346613 7128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:03:47.346772 7128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "d8602802f45db4edb2a6b8e51812b0ff"
server_key_iv: "842d3bc1ad75141e8694a72fcefe9148"
server_key_version: "encryptionkey@0"
I20260925 11:03:47.347797 7128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:47.372586 7128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:47.376617 7128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:47.376864 7128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:47.394060 7128 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:44035
I20260925 11:03:47.394083 7195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:44035 every 8 connection(s)
I20260925 11:03:47.395613 7128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:03:47.396729 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 7128
I20260925 11:03:47.397475 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:33653
--local_ip_for_outbound_sockets=127.24.159.193
--tserver_master_addrs=127.24.159.254:44035
--webserver_port=39551
--webserver_interface=127.24.159.193
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:03:47.402861 7196 sys_catalog.cc:263] Verifying existing consensus state
I20260925 11:03:47.406158 7196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Bootstrap starting.
I20260925 11:03:47.425434 7196 log.cc:824] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:47.437422 7196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 11:03:47.437913 7196 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Bootstrap complete.
I20260925 11:03:47.446743 7196 raft_consensus.cc:348] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:47.447571 7196 raft_consensus.cc:729] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b588f0d3247d4b0f9e91bbb22ecc1cb1, State: Initialized, Role: FOLLOWER
I20260925 11:03:47.448163 7196 consensus_queue.cc:260] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:47.448463 7196 raft_consensus.cc:388] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:03:47.448577 7196 raft_consensus.cc:482] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:03:47.448741 7196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 1 FOLLOWER]: Advancing to term 2
I20260925 11:03:47.452907 7196 raft_consensus.cc:504] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:47.453388 7196 leader_election.cc:302] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b588f0d3247d4b0f9e91bbb22ecc1cb1; no voters:
I20260925 11:03:47.453955 7196 leader_election.cc:288] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 11:03:47.454098 7200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 2 FOLLOWER]: Leader election won for term 2
I20260925 11:03:47.456091 7196 sys_catalog.cc:564] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:03:47.458457 7200 raft_consensus.cc:686] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 2 LEADER]: Becoming Leader. State: Replica: b588f0d3247d4b0f9e91bbb22ecc1cb1, State: Running, Role: LEADER
I20260925 11:03:47.459141 7200 consensus_queue.cc:237] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:47.461668 7201 sys_catalog.cc:455] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } } }
I20260925 11:03:47.462320 7201 sys_catalog.cc:458] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:47.462119 7202 sys_catalog.cc:455] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b588f0d3247d4b0f9e91bbb22ecc1cb1. Latest consensus state: current_term: 2 leader_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } } }
I20260925 11:03:47.462554 7202 sys_catalog.cc:458] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:47.471575 7207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:03:47.479701 7207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:03:47.481067 7207 catalog_manager.cc:1259] Loaded cluster ID: 250e0a8403004d888da948d43089d2b1
I20260925 11:03:47.481213 7207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:03:47.484205 7207 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260925 11:03:47.500950 7207 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260925 11:03:47.515466 7207 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:03:47.517274 7207 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Loaded TSK: 0
I20260925 11:03:47.519007 7207 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 11:03:47.732378 7198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:47.732656 7198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:47.732779 7198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:47.740041 7198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:47.740146 7198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:47.740280 7198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:03:47.749909 7198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:33653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=39551
--tserver_master_addrs=127.24.159.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:47.751313 7198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:47.752671 7198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:47.766685 7224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:47.766865 7223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:47.767288 7226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:47.767936 7198 server_base.cc:1034] running on GCE node
I20260925 11:03:47.768529 7198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:47.769476 7198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:47.770670 7198 hybrid_clock.cc:648] HybridClock initialized: now 1790334227770606 us; error 57 us; skew 500 ppm
I20260925 11:03:47.771003 7198 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:47 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334227, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:47.774183 7198 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:03:47.775635 7198 webserver.cc:460] Webserver started at http://127.24.159.193:39551/ using document root <none> and password file <none>
I20260925 11:03:47.776301 7198 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:47.776427 7198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:47.781901 7198 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.000s
I20260925 11:03:47.784729 7233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:47.786142 7198 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:03:47.786304 7198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "818c806154164b67b3df387895644ffb"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "12490b366fc755fc970ef608c094a56d"
server_key_iv: "be196bf8c1e1dec1760a14057428c5ea"
server_key_version: "encryptionkey@0"
I20260925 11:03:47.787246 7198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:47.803113 7198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:47.807082 7198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:47.807315 7198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:47.808265 7198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:47.809625 7198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:47.809718 7198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:47.809821 7198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:47.809892 7198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:47.840415 7198 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:33653
I20260925 11:03:47.840476 7345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:33653 every 8 connection(s)
I20260925 11:03:47.841840 7198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:03:47.848762 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 7198
I20260925 11:03:47.849467 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:39897
--local_ip_for_outbound_sockets=127.24.159.194
--tserver_master_addrs=127.24.159.254:44035
--webserver_port=42381
--webserver_interface=127.24.159.194
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:03:47 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334227, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:47.865618 7349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:47.844960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:48063)
0925 11:03:47.845767 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:47.845816 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:47.847540 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:47.849049 (+ 1509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:47.849056 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:47.849573 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:47.850185 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:47.850238 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:47.851939 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:47.851945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:47.853824 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:47.853838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:47.854853 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:47.854859 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:47.855118 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:47.855787 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:47.855813 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:47.858118 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:47.861894 (+ 3776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:47.861901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:47.861913 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:47.862368 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:47.862815 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:47.862818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:47.862820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:47.862985 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:47.863765 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:47.863770 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:47.864061 (+ 291us) client_negotiation.cc:769] Sending connection context
0925 11:03:47.864267 (+ 206us) client_negotiation.cc:240] Negotiation successful
0925 11:03:47.864662 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":314,"threads_started":1}
I20260925 11:03:47.865985 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:47.844688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:48063 (local address 127.24.159.254:44035)
0925 11:03:47.845193 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:47.845205 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:47.845920 (+ 715us) server_negotiation.cc:407] Connection header received
0925 11:03:47.848001 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:47.848014 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:47.848455 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:47.848909 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:47.850484 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:47.851777 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:47.853952 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:47.854638 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:47.854771 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:47.858402 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:47.858413 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:47.858432 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:47.858478 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:47.861765 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:47.862503 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:47.862511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:47.862519 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:47.862717 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:47.863093 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:47.863097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:47.863100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:47.863660 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:47.863838 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:47.864521 (+ 683us) server_negotiation.cc:299] Negotiation successful
0925 11:03:47.864910 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":216,"threads_started":1}
I20260925 11:03:47.868767 7346 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44035
I20260925 11:03:47.869055 7346 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:47.869771 7346 heartbeater.cc:511] Master 127.24.159.254:44035 requested a full tablet report, sending...
I20260925 11:03:47.871953 7161 ts_manager.cc:194] Registered new tserver with Master: 818c806154164b67b3df387895644ffb (127.24.159.193:33653)
I20260925 11:03:47.873467 7161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:48063
W20260925 11:03:48.167119 7350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:48.167416 7350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:48.167552 7350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:48.174837 7350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:48.174957 7350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:48.175107 7350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:03:48.184760 7350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:39897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=42381
--tserver_master_addrs=127.24.159.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:48.186041 7350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:48.187350 7350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:48.199542 7355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:48.199589 7358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:48.199556 7356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:48.200546 7350 server_base.cc:1034] running on GCE node
I20260925 11:03:48.201120 7350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:48.202004 7350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:48.203195 7350 hybrid_clock.cc:648] HybridClock initialized: now 1790334228203143 us; error 52 us; skew 500 ppm
I20260925 11:03:48.203503 7350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:48 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334228, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:48.206429 7350 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:03:48.207743 7350 webserver.cc:460] Webserver started at http://127.24.159.194:42381/ using document root <none> and password file <none>
I20260925 11:03:48.208379 7350 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:48.208508 7350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:48.213661 7350 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:03:48.216297 7365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:48.217588 7350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:03:48.217748 7350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "736b126b26374837a9e423b9e9bb69e4"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "7229be0c333e1adfa0fc4ee166b60aa9"
server_key_iv: "1a982e8f22fa40a7887d6e7d3979f512"
server_key_version: "encryptionkey@0"
I20260925 11:03:48.218725 7350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:48.247326 7350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:48.251493 7350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:48.251758 7350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:48.252818 7350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:48.254164 7350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:48.254297 7350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:48.254400 7350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:48.254477 7350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:48.284695 7350 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:39897
I20260925 11:03:48.284777 7477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:39897 every 8 connection(s)
I20260925 11:03:48.286273 7350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:03:48.294019 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 7350
I20260925 11:03:48.294607 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:37427
--local_ip_for_outbound_sockets=127.24.159.195
--tserver_master_addrs=127.24.159.254:44035
--webserver_port=33671
--webserver_interface=127.24.159.195
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:03:48 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334228, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:48.307051 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.288693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:46983 (local address 127.24.159.254:44035)
0925 11:03:48.288872 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:48.288876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:48.290064 (+ 1188us) server_negotiation.cc:407] Connection header received
0925 11:03:48.292036 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:48.292041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:48.292110 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:48.292188 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:48.293664 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.294555 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.296730 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.297178 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.297280 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.301249 (+ 3969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:48.301253 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:48.301257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:48.301290 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:48.304162 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:48.304817 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:48.304823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:48.304827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:48.304980 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:48.305344 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:48.305348 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:48.305351 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:48.305677 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:48.305812 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:48.306496 (+ 684us) server_negotiation.cc:299] Negotiation successful
0925 11:03:48.306730 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 11:03:48.307873 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.289209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:46983)
0925 11:03:48.289899 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:48.289936 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:48.291773 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:48.292396 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:48.292405 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:48.292899 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:48.293472 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.293489 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.294721 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.294727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.296558 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.296573 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.297393 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.297399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.297664 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.298361 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:48.298388 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:48.301028 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:48.304288 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:48.304295 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:48.304308 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:48.304712 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:48.305066 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:48.305069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:48.305072 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:48.305250 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:48.305779 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:48.305785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:48.306093 (+ 308us) client_negotiation.cc:769] Sending connection context
0925 11:03:48.306378 (+ 285us) client_negotiation.cc:240] Negotiation successful
0925 11:03:48.306778 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":267,"threads_started":1}
I20260925 11:03:48.309305 7478 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44035
I20260925 11:03:48.309582 7478 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:48.310340 7478 heartbeater.cc:511] Master 127.24.159.254:44035 requested a full tablet report, sending...
I20260925 11:03:48.311774 7161 ts_manager.cc:194] Registered new tserver with Master: 736b126b26374837a9e423b9e9bb69e4 (127.24.159.194:39897)
I20260925 11:03:48.312559 7161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:46983
W20260925 11:03:48.614591 7481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:48.614879 7481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:48.615034 7481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:48.622155 7481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:48.622285 7481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:48.622437 7481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:03:48.632009 7481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:37427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=33671
--tserver_master_addrs=127.24.159.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:48.633242 7481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:48.634531 7481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:48.647037 7489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:48.647097 7486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:48.647068 7487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:48.647603 7481 server_base.cc:1034] running on GCE node
I20260925 11:03:48.648221 7481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:48.649138 7481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:48.650336 7481 hybrid_clock.cc:648] HybridClock initialized: now 1790334228650287 us; error 40 us; skew 500 ppm
I20260925 11:03:48.650635 7481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:48 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334228, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:48.653735 7481 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:03:48.655158 7481 webserver.cc:460] Webserver started at http://127.24.159.195:33671/ using document root <none> and password file <none>
I20260925 11:03:48.655812 7481 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:48.655949 7481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:48.661144 7481 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 11:03:48.663759 7496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:48.665000 7481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260925 11:03:48.665146 7481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "4c01a5b49c77401ba10b36c5666591f6"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "b352d226ab2fbdd49a036a9f8817dc1d"
server_key_iv: "daac3039ba71c18b94284f893c506ea6"
server_key_version: "encryptionkey@0"
I20260925 11:03:48.666076 7481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:48.682585 7481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:48.686537 7481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:48.686772 7481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:48.687677 7481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:48.688920 7481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:48.689019 7481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:48.689116 7481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:48.689186 7481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:48.718318 7481 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:37427
I20260925 11:03:48.718415 7608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:37427 every 8 connection(s)
I20260925 11:03:48.719710 7481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:03:48.728807 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 7481
Sep 25 11:03:48 dist-test-slave-trms krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334228, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:03:48.740535 7348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.722108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:51033 (local address 127.24.159.254:44035)
0925 11:03:48.722307 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:48.722312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:48.723562 (+ 1250us) server_negotiation.cc:407] Connection header received
0925 11:03:48.725358 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:48.725364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:48.725445 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:48.725564 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:48.727177 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.727874 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.729425 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.730136 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.730311 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.734324 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:48.734330 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:48.734335 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:48.734378 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:48.737529 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:48.738243 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:48.738249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:48.738253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:48.738425 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:48.738822 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:48.738827 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:48.738830 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:48.739161 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:48.739275 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:48.740111 (+ 836us) server_negotiation.cc:299] Negotiation successful
0925 11:03:48.740289 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:03:48.741956 7611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.722657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:51033)
0925 11:03:48.723399 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:48.723438 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:48.725082 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:48.725752 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:48.725761 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:48.726403 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:03:48.727020 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.727039 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.727986 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.727989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.729291 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.729303 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.730405 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.730412 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.730710 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.731386 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:03:48.731415 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:03:48.734105 (+ 2690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:03:48.737659 (+ 3554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:48.737666 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:48.737679 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:48.738088 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:48.738518 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:03:48.738521 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:03:48.738523 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:03:48.738709 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:03:48.739277 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:03:48.739286 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:03:48.739724 (+ 438us) client_negotiation.cc:769] Sending connection context
0925 11:03:48.740009 (+ 285us) client_negotiation.cc:240] Negotiation successful
0925 11:03:48.740540 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":298,"threads_started":1}
I20260925 11:03:48.743492 7609 heartbeater.cc:347] Connected to a master server at 127.24.159.254:44035
I20260925 11:03:48.743795 7609 heartbeater.cc:464] Registering TS with master...
I20260925 11:03:48.744516 7609 heartbeater.cc:511] Master 127.24.159.254:44035 requested a full tablet report, sending...
I20260925 11:03:48.744930 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.729645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36414 (local address 127.24.159.254:44035)
0925 11:03:48.730115 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:48.730120 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:48.730645 (+ 525us) server_negotiation.cc:407] Connection header received
0925 11:03:48.730819 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:48.730823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:48.730876 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:48.730954 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:48.732840 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.733840 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.735673 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.736222 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.736323 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.739198 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:48.739203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:48.739206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:48.739242 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:48.742311 (+ 3069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:48.743068 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:48.743073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:48.743076 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:48.743219 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:48.743587 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:48.743591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:48.743594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:48.743855 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:48.744094 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:48.744393 (+ 299us) server_negotiation.cc:299] Negotiation successful
0925 11:03:48.744596 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":259,"threads_started":1}
I20260925 11:03:48.745924 7161 ts_manager.cc:194] Registered new tserver with Master: 4c01a5b49c77401ba10b36c5666591f6 (127.24.159.195:37427)
I20260925 11:03:48.746779 7161 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:51033
I20260925 11:03:48.747311 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:03:48.753805 7612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.748144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36420 (local address 127.24.159.254:44035)
0925 11:03:48.748282 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:48.748286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:48.748793 (+ 507us) server_negotiation.cc:407] Connection header received
0925 11:03:48.749003 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:48.749007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:48.749129 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:48.749247 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:03:48.749647 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.750388 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.751914 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.752317 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.752439 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.752870 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:03:48.753291 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:03:48.753423 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:48.753513 (+ 90us) server_negotiation.cc:299] Negotiation successful
0925 11:03:48.753585 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260925 11:03:48.754592 7161 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:03:48.761745 7161 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:03:48.796506 7620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.772961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39897 (local address 127.0.0.1:45050)
0925 11:03:48.786559 (+ 13598us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:48.786589 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:48.786725 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:48.789049 (+ 2324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:48.789057 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:48.789105 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:48.789545 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.789557 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.791537 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.791542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.793661 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.793673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.795481 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.795487 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.795693 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.795754 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 11:03:48.795883 (+ 129us) client_negotiation.cc:240] Negotiation successful
0925 11:03:48.796068 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13464,"mutex_wait_us":21,"thread_start_us":14830,"threads_started":1}
I20260925 11:03:48.796545 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.773622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45050 (local address 127.24.159.194:39897)
0925 11:03:48.784595 (+ 10973us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:48.784605 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:48.786859 (+ 2254us) server_negotiation.cc:407] Connection header received
0925 11:03:48.786988 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:48.786997 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:48.787313 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:48.787515 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:48.789794 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.791375 (+ 1581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.793808 (+ 2433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.794950 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.795133 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.795320 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:48.795938 (+ 618us) server_negotiation.cc:299] Negotiation successful
0925 11:03:48.796115 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10771,"thread_start_us":10639,"threads_started":1}
I20260925 11:03:48.799430 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.774225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59188 (local address 127.24.159.193:33653)
0925 11:03:48.790183 (+ 15958us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:48.790385 (+ 202us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:48.790975 (+ 590us) server_negotiation.cc:407] Connection header received
0925 11:03:48.791137 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:48.791145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:48.791448 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:48.791658 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:48.792334 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.793702 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.796752 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.798043 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.798185 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.798370 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:48.798963 (+ 593us) server_negotiation.cc:299] Negotiation successful
0925 11:03:48.799130 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15773,"thread_start_us":15970,"threads_started":1}
I20260925 11:03:48.799659 7622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.772809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:37427 (local address 127.0.0.1:49184)
0925 11:03:48.782749 (+ 9940us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:48.783419 (+ 670us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:48.783648 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:48.792379 (+ 8731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:48.792383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:48.792414 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:48.792782 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.792793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.794692 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.794698 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.797067 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.797081 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.798847 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.798854 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.799015 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.799045 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 11:03:48.799121 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 11:03:48.799248 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9679,"mutex_wait_us":1,"thread_start_us":18271,"threads_started":1}
I20260925 11:03:48.800153 7619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.773525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:33653 (local address 127.0.0.1:59188)
0925 11:03:48.790877 (+ 17352us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:48.790896 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:48.791009 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:48.791656 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:48.791660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:48.791690 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:48.792115 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.792125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.793865 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.793870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.796557 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.796573 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.798574 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.798580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:48.798752 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.798789 (+ 37us) client_negotiation.cc:769] Sending connection context
0925 11:03:48.799670 (+ 881us) client_negotiation.cc:240] Negotiation successful
0925 11:03:48.799792 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17235,"mutex_wait_us":3,"thread_start_us":8823,"threads_started":1}
I20260925 11:03:48.800648 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.773109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.24.159.195:37427)
0925 11:03:48.790874 (+ 17765us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:48.790881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:48.790914 (+ 33us) server_negotiation.cc:407] Connection header received
0925 11:03:48.791918 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:48.791927 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:48.792164 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:48.792411 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:48.792979 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.794534 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.797227 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.798633 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.798806 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:48.800097 (+ 1291us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:48.800172 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 11:03:48.800329 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17621,"thread_start_us":17470,"threads_started":1}
I20260925 11:03:48.800414 7413 tablet_service.cc:1467] Processing CreateTablet for tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad (DEFAULT_TABLE table=test-table [id=481f63ff7b4f472e989715e10c9dad2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:03:48.801998 7413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:48.803364 7544 tablet_service.cc:1467] Processing CreateTablet for tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad (DEFAULT_TABLE table=test-table [id=481f63ff7b4f472e989715e10c9dad2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:03:48.803766 7281 tablet_service.cc:1467] Processing CreateTablet for tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad (DEFAULT_TABLE table=test-table [id=481f63ff7b4f472e989715e10c9dad2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:03:48.804610 7544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:48.805028 7281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:03:48.815850 7625 tablet_bootstrap.cc:492] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: Bootstrap starting.
I20260925 11:03:48.819700 7627 tablet_bootstrap.cc:492] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: Bootstrap starting.
I20260925 11:03:48.820075 7625 tablet_bootstrap.cc:654] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:48.821408 7625 log.cc:824] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:48.822760 7626 tablet_bootstrap.cc:492] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: Bootstrap starting.
I20260925 11:03:48.823151 7627 tablet_bootstrap.cc:654] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:48.824412 7627 log.cc:824] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:48.824880 7625 tablet_bootstrap.cc:492] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: No bootstrap required, opened a new log
I20260925 11:03:48.825199 7625 ts_tablet_manager.cc:1397] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260925 11:03:48.827477 7627 tablet_bootstrap.cc:492] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: No bootstrap required, opened a new log
I20260925 11:03:48.827488 7626 tablet_bootstrap.cc:654] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: Neither blocks nor log segments found. Creating new log.
I20260925 11:03:48.827762 7627 ts_tablet_manager.cc:1397] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260925 11:03:48.828665 7626 log.cc:824] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:48.831665 7626 tablet_bootstrap.cc:492] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: No bootstrap required, opened a new log
I20260925 11:03:48.831933 7626 ts_tablet_manager.cc:1397] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260925 11:03:48.833077 7625 raft_consensus.cc:348] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:48.833415 7625 raft_consensus.cc:374] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:48.833509 7625 raft_consensus.cc:729] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c01a5b49c77401ba10b36c5666591f6, State: Initialized, Role: FOLLOWER
I20260925 11:03:48.834048 7625 consensus_queue.cc:260] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:48.835090 7609 heartbeater.cc:503] Master 127.24.159.254:44035 was elected leader, sending a full tablet report...
I20260925 11:03:48.835922 7625 ts_tablet_manager.cc:1428] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260925 11:03:48.837608 7627 raft_consensus.cc:348] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:48.838054 7627 raft_consensus.cc:374] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:48.838146 7627 raft_consensus.cc:729] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 818c806154164b67b3df387895644ffb, State: Initialized, Role: FOLLOWER
I20260925 11:03:48.838804 7627 consensus_queue.cc:260] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:48.840054 7346 heartbeater.cc:503] Master 127.24.159.254:44035 was elected leader, sending a full tablet report...
I20260925 11:03:48.843698 7626 raft_consensus.cc:348] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:48.844099 7626 raft_consensus.cc:374] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:03:48.844216 7626 raft_consensus.cc:729] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 736b126b26374837a9e423b9e9bb69e4, State: Initialized, Role: FOLLOWER
I20260925 11:03:48.844818 7626 consensus_queue.cc:260] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:48.845796 7478 heartbeater.cc:503] Master 127.24.159.254:44035 was elected leader, sending a full tablet report...
I20260925 11:03:48.846417 7626 ts_tablet_manager.cc:1428] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260925 11:03:48.847486 7627 ts_tablet_manager.cc:1428] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: Time spent starting tablet: real 0.020s user 0.010s sys 0.003s
W20260925 11:03:48.971210 7610 tablet.cc:2369] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:03:48.994336 7632 raft_consensus.cc:482] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:03:48.994668 7632 raft_consensus.cc:504] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:48.996328 7632 leader_election.cc:288] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4c01a5b49c77401ba10b36c5666591f6 (127.24.159.195:37427), 736b126b26374837a9e423b9e9bb69e4 (127.24.159.194:39897)
I20260925 11:03:49.002604 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.996751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:54463 (local address 127.24.159.194:39897)
0925 11:03:48.996963 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:48.996969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:48.997407 (+ 438us) server_negotiation.cc:407] Connection header received
0925 11:03:48.997629 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:48.997635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:48.997714 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:48.997780 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:48.998353 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.999082 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.000862 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.001658 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.001778 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:49.001833 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:49.002229 (+ 396us) server_negotiation.cc:299] Negotiation successful
0925 11:03:49.002356 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:03:49.002611 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.996707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:50503 (local address 127.24.159.195:37427)
0925 11:03:48.996919 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:48.996925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:48.997555 (+ 630us) server_negotiation.cc:407] Connection header received
0925 11:03:48.997701 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:48.997705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:48.997766 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:48.997849 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:03:48.998559 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.999307 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.000968 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.001771 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.001927 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:49.001997 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:49.002270 (+ 273us) server_negotiation.cc:299] Negotiation successful
0925 11:03:49.002389 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 11:03:49.002655 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.996625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:39897 (local address 127.24.159.193:54463)
0925 11:03:48.997283 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:48.997303 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:48.997464 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:48.997817 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:48.997822 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:48.997847 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:48.998215 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.998224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.999215 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.999220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:49.000717 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:49.000731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.001856 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.001862 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:49.002055 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:49.002078 (+ 23us) client_negotiation.cc:769] Sending connection context
0925 11:03:49.002149 (+ 71us) client_negotiation.cc:240] Negotiation successful
0925 11:03:49.002284 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":342,"threads_started":1}
I20260925 11:03:49.003127 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:48.996723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:37427 (local address 127.24.159.193:50503)
0925 11:03:48.997451 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:48.997465 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:03:48.997586 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:03:48.997887 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:03:48.997892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:03:48.997968 (+ 76us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:03:48.998404 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:48.998418 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:48.999427 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:48.999431 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:49.000807 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:03:49.000822 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.001973 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.001977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:03:49.002100 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:49.002121 (+ 21us) client_negotiation.cc:769] Sending connection context
0925 11:03:49.002234 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 11:03:49.002355 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":3,"spinlock_wait_cycles":18688,"thread_start_us":433,"threads_started":1}
I20260925 11:03:49.003666 7433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfbb31f7e8ff4c0e8218c96d0b18a9ad" candidate_uuid: "818c806154164b67b3df387895644ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736b126b26374837a9e423b9e9bb69e4" is_pre_election: true
I20260925 11:03:49.004041 7564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfbb31f7e8ff4c0e8218c96d0b18a9ad" candidate_uuid: "818c806154164b67b3df387895644ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c01a5b49c77401ba10b36c5666591f6" is_pre_election: true
I20260925 11:03:49.004252 7433 raft_consensus.cc:2454] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 818c806154164b67b3df387895644ffb in term 0.
I20260925 11:03:49.004558 7564 raft_consensus.cc:2454] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 818c806154164b67b3df387895644ffb in term 0.
I20260925 11:03:49.004945 7237 leader_election.cc:302] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 736b126b26374837a9e423b9e9bb69e4, 818c806154164b67b3df387895644ffb; no voters:
I20260925 11:03:49.005303 7632 raft_consensus.cc:2781] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:03:49.005455 7632 raft_consensus.cc:482] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:03:49.005551 7632 raft_consensus.cc:3037] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:49.007635 7632 raft_consensus.cc:504] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:49.008133 7632 leader_election.cc:288] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [CANDIDATE]: Term 1 election: Requested vote from peers 4c01a5b49c77401ba10b36c5666591f6 (127.24.159.195:37427), 736b126b26374837a9e423b9e9bb69e4 (127.24.159.194:39897)
I20260925 11:03:49.008551 7564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfbb31f7e8ff4c0e8218c96d0b18a9ad" candidate_uuid: "818c806154164b67b3df387895644ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4c01a5b49c77401ba10b36c5666591f6"
I20260925 11:03:49.008577 7433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfbb31f7e8ff4c0e8218c96d0b18a9ad" candidate_uuid: "818c806154164b67b3df387895644ffb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "736b126b26374837a9e423b9e9bb69e4"
I20260925 11:03:49.008752 7564 raft_consensus.cc:3037] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:49.008772 7433 raft_consensus.cc:3037] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:03:49.011824 7564 raft_consensus.cc:2454] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 818c806154164b67b3df387895644ffb in term 1.
I20260925 11:03:49.011824 7433 raft_consensus.cc:2454] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 818c806154164b67b3df387895644ffb in term 1.
I20260925 11:03:49.012223 7235 leader_election.cc:302] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4c01a5b49c77401ba10b36c5666591f6, 736b126b26374837a9e423b9e9bb69e4, 818c806154164b67b3df387895644ffb; no voters:
I20260925 11:03:49.012581 7632 raft_consensus.cc:2781] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:03:49.012996 7632 raft_consensus.cc:686] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 1 LEADER]: Becoming Leader. State: Replica: 818c806154164b67b3df387895644ffb, State: Running, Role: LEADER
I20260925 11:03:49.013494 7632 consensus_queue.cc:237] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } }
I20260925 11:03:49.017807 7160 catalog_manager.cc:5495] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb reported cstate change: term changed from 0 to 1, leader changed from <none> to 818c806154164b67b3df387895644ffb (127.24.159.193). New cstate: current_term: 1 leader_uuid: "818c806154164b67b3df387895644ffb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "818c806154164b67b3df387895644ffb" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 33653 } health_report { overall_health: HEALTHY } } }
I20260925 11:03:49.033563 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:49.028304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59204 (local address 127.24.159.193:33653)
0925 11:03:49.028485 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:49.028491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:49.028556 (+ 65us) server_negotiation.cc:407] Connection header received
0925 11:03:49.028781 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:49.028785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:49.028861 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:49.028997 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:03:49.029364 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.030042 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.031509 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.031926 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.032033 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:49.032501 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:03:49.032996 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:03:49.033144 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:49.033218 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 11:03:49.033298 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 11:03:49.038049 7479 tablet.cc:2369] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:03:49.041323 7433 raft_consensus.cc:1261] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [term 1 FOLLOWER]: Refusing update from remote peer 818c806154164b67b3df387895644ffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:03:49.041388 7564 raft_consensus.cc:1261] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [term 1 FOLLOWER]: Refusing update from remote peer 818c806154164b67b3df387895644ffb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:03:49.042294 7632 consensus_queue.cc:1035] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "736b126b26374837a9e423b9e9bb69e4" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 39897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:03:49.042569 7636 consensus_queue.cc:1035] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c01a5b49c77401ba10b36c5666591f6" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 37427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:03:49.068912 7161 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:03:49.069206 7161 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:03:49.072795 7161 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Sending DeleteTablet for 3 replicas of tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad
I20260925 11:03:49.073844 7413 tablet_service.cc:1514] Processing DeleteTablet for tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:49 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:45050
I20260925 11:03:49.073941 7544 tablet_service.cc:1514] Processing DeleteTablet for tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:49 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:49184
I20260925 11:03:49.074908 7281 tablet_service.cc:1514] Processing DeleteTablet for tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:03:49 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:59188
I20260925 11:03:49.075017 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 7198
I20260925 11:03:49.075944 7650 tablet_replica.cc:330] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: stopping tablet replica
I20260925 11:03:49.076355 7650 raft_consensus.cc:2229] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:03:49.076987 7650 raft_consensus.cc:2258] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:03:49.077700 7648 tablet_replica.cc:330] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: stopping tablet replica
I20260925 11:03:49.078082 7648 raft_consensus.cc:2229] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:03:49.078563 7648 raft_consensus.cc:2258] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:03:49.080292 7649 tablet_replica.cc:330] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: stopping tablet replica
I20260925 11:03:49.080623 7650 ts_tablet_manager.cc:1905] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:03:49.080681 7649 raft_consensus.cc:2229] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:03:49.081130 7649 raft_consensus.cc:2258] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:03:49.084260 7649 ts_tablet_manager.cc:1905] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:03:49.088131 7650 ts_tablet_manager.cc:1918] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:03:49.088294 7650 log.cc:1196] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/bfbb31f7e8ff4c0e8218c96d0b18a9ad
I20260925 11:03:49.088855 7650 ts_tablet_manager.cc:1939] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 818c806154164b67b3df387895644ffb: Deleting consensus metadata
I20260925 11:03:49.090778 7145 catalog_manager.cc:4841] TS 818c806154164b67b3df387895644ffb (127.24.159.193:33653): tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad (table test-table [id=481f63ff7b4f472e989715e10c9dad2a]) successfully deleted
I20260925 11:03:49.092005 7624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:49.076723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59212 (local address 127.24.159.193:33653)
0925 11:03:49.076904 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:49.076911 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:49.076929 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:03:49.076991 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:49.076996 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:49.077076 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:49.077170 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:49.079056 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.079860 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.081649 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.082141 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.082327 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:49.085547 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:49.085558 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:49.085574 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:49.085615 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:49.088715 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:49.089589 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:49.089596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:49.089598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:49.089760 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:49.090144 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:49.090149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:49.090152 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:49.090570 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:49.090908 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:49.091408 (+ 500us) server_negotiation.cc:299] Negotiation successful
0925 11:03:49.091626 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:03:49.092372 7648 ts_tablet_manager.cc:1905] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:03:49.092687 7341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:49.097821 7649 ts_tablet_manager.cc:1918] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:03:49.097990 7649 log.cc:1196] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/bfbb31f7e8ff4c0e8218c96d0b18a9ad
I20260925 11:03:49.098532 7649 ts_tablet_manager.cc:1939] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 4c01a5b49c77401ba10b36c5666591f6: Deleting consensus metadata
I20260925 11:03:49.099438 7648 ts_tablet_manager.cc:1918] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:03:49.099583 7648 log.cc:1196] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/bfbb31f7e8ff4c0e8218c96d0b18a9ad
I20260925 11:03:49.100070 7648 ts_tablet_manager.cc:1939] T bfbb31f7e8ff4c0e8218c96d0b18a9ad P 736b126b26374837a9e423b9e9bb69e4: Deleting consensus metadata
I20260925 11:03:49.100167 7146 catalog_manager.cc:4841] TS 4c01a5b49c77401ba10b36c5666591f6 (127.24.159.195:37427): tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad (table test-table [id=481f63ff7b4f472e989715e10c9dad2a]) successfully deleted
I20260925 11:03:49.101580 7148 catalog_manager.cc:4841] TS 736b126b26374837a9e423b9e9bb69e4 (127.24.159.194:39897): tablet bfbb31f7e8ff4c0e8218c96d0b18a9ad (table test-table [id=481f63ff7b4f472e989715e10c9dad2a]) successfully deleted
I20260925 11:03:49.189971 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 7198
I20260925 11:03:49.204614 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 7350
I20260925 11:03:49.217370 7623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:49.205409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45054 (local address 127.24.159.194:39897)
0925 11:03:49.205593 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:49.205598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:49.205618 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:03:49.205727 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:49.205732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:49.205795 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:49.205927 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:49.207222 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.207972 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.209278 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.209686 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.209794 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:49.212189 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:49.212202 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:49.212218 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:49.212254 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:49.214701 (+ 2447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:49.215405 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:49.215413 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:49.215415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:49.215535 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:49.215885 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:49.215889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:49.215891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:49.216255 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:49.216542 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:49.216825 (+ 283us) server_negotiation.cc:299] Negotiation successful
0925 11:03:49.217042 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:03:49.218106 7473 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:49.306141 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 7350
I20260925 11:03:49.320068 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 7481
I20260925 11:03:49.332737 7621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:49.320812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.24.159.195:37427)
0925 11:03:49.320994 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 11:03:49.320998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:03:49.321017 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:03:49.321153 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:03:49.321157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:03:49.321228 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:03:49.321307 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:03:49.322676 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.323397 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.324670 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:03:49.325059 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:03:49.325151 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:03:49.327545 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:03:49.327561 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:03:49.327589 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:03:49.327625 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:03:49.330328 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:49.330989 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:49.330997 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:49.330999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:49.331140 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:03:49.331430 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:03:49.331432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:03:49.331434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:03:49.331779 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:03:49.331990 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0925 11:03:49.332260 (+ 270us) server_negotiation.cc:299] Negotiation successful
0925 11:03:49.332425 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:03:49.333379 7604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:49.421463 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 7481
I20260925 11:03:49.435940 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 7128
I20260925 11:03:49.437067 7191 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:03:49.548326 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 7128
I20260925 11:03:49.565847 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44035
--webserver_interface=127.24.159.254
--webserver_port=42961
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:44035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:03:49.880391 7655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:49.880708 7655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:49.880841 7655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:49.888106 7655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:03:49.888226 7655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:49.888311 7655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:49.888384 7655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:03:49.888444 7655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:03:49.898357 7655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:44035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=42961
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:49.899674 7655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:49.901001 7655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:49.914601 7661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:49.914676 7663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:49.914659 7660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:49.915467 7655 server_base.cc:1034] running on GCE node
I20260925 11:03:49.916302 7655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:49.917766 7655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:49.918999 7655 hybrid_clock.cc:648] HybridClock initialized: now 1790334229918956 us; error 29 us; skew 500 ppm
I20260925 11:03:49.919376 7655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:49 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334229, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:49.922395 7655 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:03:49.923789 7655 webserver.cc:460] Webserver started at http://127.24.159.254:42961/ using document root <none> and password file <none>
I20260925 11:03:49.924448 7655 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:49.924546 7655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:49.929847 7655 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:03:49.932564 7670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:49.933844 7655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 11:03:49.934012 7655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "d8602802f45db4edb2a6b8e51812b0ff"
server_key_iv: "842d3bc1ad75141e8694a72fcefe9148"
server_key_version: "encryptionkey@0"
I20260925 11:03:49.934990 7655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:49.964656 7655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:49.968469 7655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:49.968709 7655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:49.985585 7655 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:44035
I20260925 11:03:49.985605 7722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:44035 every 8 connection(s)
I20260925 11:03:49.987035 7655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:03:49.993598 7723 sys_catalog.cc:263] Verifying existing consensus state
I20260925 11:03:49.996621 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 7655
I20260925 11:03:49.997006 7723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Bootstrap starting.
I20260925 11:03:49.997584 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:33653
--local_ip_for_outbound_sockets=127.24.159.193
--tserver_master_addrs=127.24.159.254:44035
--webserver_port=39551
--webserver_interface=127.24.159.193
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:03:50.018644 7723 log.cc:824] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Log is configured to *not* fsync() on all Append() calls
I20260925 11:03:50.036911 7723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 11:03:50.037416 7723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: Bootstrap complete.
I20260925 11:03:50.046474 7723 raft_consensus.cc:348] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:50.047333 7723 raft_consensus.cc:729] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b588f0d3247d4b0f9e91bbb22ecc1cb1, State: Initialized, Role: FOLLOWER
I20260925 11:03:50.047933 7723 consensus_queue.cc:260] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:50.048139 7723 raft_consensus.cc:388] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:03:50.048251 7723 raft_consensus.cc:482] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:03:50.048403 7723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 2 FOLLOWER]: Advancing to term 3
I20260925 11:03:50.052325 7723 raft_consensus.cc:504] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:50.052799 7723 leader_election.cc:302] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b588f0d3247d4b0f9e91bbb22ecc1cb1; no voters:
I20260925 11:03:50.053383 7723 leader_election.cc:288] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 11:03:50.053527 7727 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 3 FOLLOWER]: Leader election won for term 3
I20260925 11:03:50.055513 7723 sys_catalog.cc:564] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:03:50.057192 7727 raft_consensus.cc:686] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [term 3 LEADER]: Becoming Leader. State: Replica: b588f0d3247d4b0f9e91bbb22ecc1cb1, State: Running, Role: LEADER
I20260925 11:03:50.057724 7727 consensus_queue.cc:237] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } }
I20260925 11:03:50.060140 7728 sys_catalog.cc:455] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } } }
I20260925 11:03:50.060482 7728 sys_catalog.cc:458] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:50.065439 7729 sys_catalog.cc:455] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b588f0d3247d4b0f9e91bbb22ecc1cb1. Latest consensus state: current_term: 3 leader_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b588f0d3247d4b0f9e91bbb22ecc1cb1" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 44035 } } }
I20260925 11:03:50.065707 7729 sys_catalog.cc:458] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:03:50.073967 7743 catalog_manager.cc:1259] Loaded cluster ID: 250e0a8403004d888da948d43089d2b1
I20260925 11:03:50.074054 7743 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: loading cluster ID for follower catalog manager: success
W20260925 11:03:50.075978 7743 catalog_manager.cc:1576] T 00000000000000000000000000000000 P b588f0d3247d4b0f9e91bbb22ecc1cb1: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260925 11:03:50.076100 7743 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260925 11:03:50.076220 7734 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:03:50.080199 7734 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:03:50.081372 7734 catalog_manager.cc:1259] Loaded cluster ID: 250e0a8403004d888da948d43089d2b1
I20260925 11:03:50.081472 7734 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260925 11:03:50.083002 7734 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7febbedf89ad google::LogMessage::Fail() at ??:0
@ 0x7febbedfb88a google::LogMessage::SendToLog() at ??:0
@ 0x7febbedf84de google::LogMessage::Flush() at ??:0
@ 0x7febbedf9a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7febd482f512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7febd482c976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7febd48b4610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7febd48b4439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7febc1a44345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7febc1a4791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7febc1a47799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7febc1a22eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7febc4cfc6db start_thread at ??:0
@ 0x7febbdc9261f clone at ??:0
W20260925 11:03:50.368937 7725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:50.369235 7725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:50.369360 7725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:50.376628 7725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:50.376750 7725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:50.376924 7725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:03:50.386672 7725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:33653
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=39551
--tserver_master_addrs=127.24.159.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:50.387984 7725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:50.389331 7725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:50.401706 7764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:50.401710 7765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:50.401837 7767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:50.402582 7725 server_base.cc:1034] running on GCE node
I20260925 11:03:50.403124 7725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:50.403967 7725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:50.405153 7725 hybrid_clock.cc:648] HybridClock initialized: now 1790334230405104 us; error 43 us; skew 500 ppm
I20260925 11:03:50.405433 7725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:50 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334230, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:50.408327 7725 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:03:50.409698 7725 webserver.cc:460] Webserver started at http://127.24.159.193:39551/ using document root <none> and password file <none>
I20260925 11:03:50.410382 7725 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:50.410499 7725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:50.415627 7725 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260925 11:03:50.418174 7774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:50.419473 7725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:03:50.419646 7725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "818c806154164b67b3df387895644ffb"
format_stamp: "Formatted at 2026-09-25 11:03:44 on dist-test-slave-trms"
server_key: "12490b366fc755fc970ef608c094a56d"
server_key_iv: "be196bf8c1e1dec1760a14057428c5ea"
server_key_version: "encryptionkey@0"
I20260925 11:03:50.420607 7725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:50.452666 7725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:50.456673 7725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:50.456925 7725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:50.457880 7725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:50.459227 7725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:50.459331 7725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:50.459435 7725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:50.459510 7725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:50.489135 7725 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:33653
I20260925 11:03:50.489197 7888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:33653 every 8 connection(s)
I20260925 11:03:50.490571 7725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:03:50.492622 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 7725
I20260925 11:03:50.493163 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:39897
--local_ip_for_outbound_sockets=127.24.159.194
--tserver_master_addrs=127.24.159.254:44035
--webserver_port=42381
--webserver_interface=127.24.159.194
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:03:50.495507 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:50.493495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:56955)
0925 11:03:50.494322 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:50.494602 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":299,"threads_started":1}
W20260925 11:03:50.496095 7889 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:44035 (0 consecutive failures): Network error: Failed to ping master at 127.24.159.254:44035: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
I20260925 11:03:50.496968 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:50.496646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:54603)
0925 11:03:50.496793 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:50.496862 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 11:03:50.497761 7891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:50.497471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:38411)
0925 11:03:50.497578 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:50.497656 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 11:03:50.497979 7889 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 11:03:50.804656 7892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:50.804919 7892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:50.805020 7892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:50.812208 7892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:50.812312 7892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:50.812422 7892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:03:50.821545 7892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:39897
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=42381
--tserver_master_addrs=127.24.159.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:50.822784 7892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:50.824076 7892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:50.836216 7897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:50.836167 7898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:50.836167 7900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:50.839535 7892 server_base.cc:1034] running on GCE node
I20260925 11:03:50.840095 7892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:50.841012 7892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:50.842233 7892 hybrid_clock.cc:648] HybridClock initialized: now 1790334230842155 us; error 44 us; skew 500 ppm
I20260925 11:03:50.842527 7892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:50 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334230, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:50.845515 7892 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:03:50.847010 7892 webserver.cc:460] Webserver started at http://127.24.159.194:42381/ using document root <none> and password file <none>
I20260925 11:03:50.847687 7892 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:50.847798 7892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:50.853003 7892 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260925 11:03:50.855708 7907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:50.857031 7892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:03:50.857189 7892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "736b126b26374837a9e423b9e9bb69e4"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "7229be0c333e1adfa0fc4ee166b60aa9"
server_key_iv: "1a982e8f22fa40a7887d6e7d3979f512"
server_key_version: "encryptionkey@0"
I20260925 11:03:50.858114 7892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:50.873459 7892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:50.877462 7892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:50.877717 7892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:50.878691 7892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:50.879942 7892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:50.880033 7892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:50.880146 7892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:50.880224 7892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:50.910115 7892 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:39897
I20260925 11:03:50.910168 8020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:39897 every 8 connection(s)
I20260925 11:03:50.911593 7892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:03:50.914737 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 7892
I20260925 11:03:50.915297 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:37427
--local_ip_for_outbound_sockets=127.24.159.195
--tserver_master_addrs=127.24.159.254:44035
--webserver_port=33671
--webserver_interface=127.24.159.195
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:03:50.916409 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:50.914473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:48357)
0925 11:03:50.915245 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:50.915498 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":303,"threads_started":1}
W20260925 11:03:50.916978 8021 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:44035 (0 consecutive failures): Network error: Failed to ping master at 127.24.159.254:44035: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
I20260925 11:03:50.917819 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:50.917491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:56717)
0925 11:03:50.917638 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:50.917719 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260925 11:03:50.918617 8023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:50.918339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:41655)
0925 11:03:50.918445 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:50.918517 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 11:03:50.918816 8021 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 11:03:51.225951 8024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:03:51.226243 8024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:03:51.226353 8024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:03:51.233364 8024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:03:51.233469 8024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:03:51.233593 8024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:03:51.243048 8024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:37427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=33671
--tserver_master_addrs=127.24.159.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:03:51.244231 8024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:03:51.245455 8024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:03:51.257702 8029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:51.257730 8032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:03:51.257637 8030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:03:51.258704 8024 server_base.cc:1034] running on GCE node
I20260925 11:03:51.259238 8024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:03:51.260087 8024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:03:51.261297 8024 hybrid_clock.cc:648] HybridClock initialized: now 1790334231261236 us; error 47 us; skew 500 ppm
I20260925 11:03:51.261583 8024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:03:51 dist-test-slave-trms krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334231, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:03:51.264595 8024 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:03:51.265920 8024 webserver.cc:460] Webserver started at http://127.24.159.195:33671/ using document root <none> and password file <none>
I20260925 11:03:51.266593 8024 fs_manager.cc:359] Metadata directory not provided
I20260925 11:03:51.266716 8024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:03:51.271957 8024 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 11:03:51.274667 8039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:51.275934 8024 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:03:51.276078 8024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "4c01a5b49c77401ba10b36c5666591f6"
format_stamp: "Formatted at 2026-09-25 11:03:45 on dist-test-slave-trms"
server_key: "b352d226ab2fbdd49a036a9f8817dc1d"
server_key_iv: "daac3039ba71c18b94284f893c506ea6"
server_key_version: "encryptionkey@0"
I20260925 11:03:51.277029 8024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:03:51.311010 8024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:03:51.315212 8024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:03:51.315465 8024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:03:51.316445 8024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:03:51.317735 8024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:03:51.317828 8024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:51.317935 8024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:03:51.318002 8024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:03:51.346568 8024 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:37427
I20260925 11:03:51.346612 8152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:37427 every 8 connection(s)
I20260925 11:03:51.347958 8024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:03:51.352752 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:51.350958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:58379)
0925 11:03:51.351664 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:51.351888 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":263,"threads_started":1}
W20260925 11:03:51.353348 8153 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:44035 (0 consecutive failures): Network error: Failed to ping master at 127.24.159.254:44035: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
I20260925 11:03:51.354122 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:51.353811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:51655)
0925 11:03:51.353959 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:51.354022 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:03:51.354878 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:51.354583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:38177)
0925 11:03:51.354710 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:51.354780 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 11:03:51.355119 8153 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 11:03:51.355764 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 8024
I20260925 11:03:51.499290 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:51.498545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:50403)
0925 11:03:51.498964 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:51.499075 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":198,"threads_started":1}
I20260925 11:03:51.920079 8158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:51.919289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:56081)
0925 11:03:51.919730 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:51.919857 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":213,"threads_started":1}
I20260925 11:03:52.356541 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:52.355703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:60647)
0925 11:03:52.356201 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:52.356322 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":257,"threads_started":1}
I20260925 11:03:52.500901 8160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:52.500152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:41997)
0925 11:03:52.500626 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:52.500737 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":219,"threads_started":1}
I20260925 11:03:52.921765 8161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:52.921032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:59619)
0925 11:03:52.921507 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:52.921597 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":237,"threads_started":1}
I20260925 11:03:53.358124 8162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:53.357437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:35135)
0925 11:03:53.357870 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:53.357959 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":210,"threads_started":1}
I20260925 11:03:53.502400 8163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:53.501695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:53169)
0925 11:03:53.502113 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:53.502225 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":213,"threads_started":1}
I20260925 11:03:53.923496 8164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:53.922770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:38461)
0925 11:03:53.923211 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:53.923293 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":232,"threads_started":1}
I20260925 11:03:54.359628 8165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:54.358948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:35457)
0925 11:03:54.359379 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:54.359461 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":221,"threads_started":1}
I20260925 11:03:54.503978 8166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:54.503261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:47357)
0925 11:03:54.503704 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:54.503790 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":215,"threads_started":1}
I20260925 11:03:54.925004 8167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:54.924299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:39663)
0925 11:03:54.924741 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:54.924819 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":228,"threads_started":1}
I20260925 11:03:55.361470 8168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:55.360613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:52737)
0925 11:03:55.361193 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:55.361273 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":265,"threads_started":1}
I20260925 11:03:55.505594 8169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:55.504879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:38569)
0925 11:03:55.505336 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:55.505416 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":237,"threads_started":1}
I20260925 11:03:55.926637 8170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:55.925901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:52329)
0925 11:03:55.926387 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:55.926467 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":229,"threads_started":1}
I20260925 11:03:56.363083 8171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:56.362379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:59395)
0925 11:03:56.362829 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:56.362913 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":234,"threads_started":1}
I20260925 11:03:56.507189 8172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:56.506460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:33463)
0925 11:03:56.506926 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:56.507024 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":230,"threads_started":1}
I20260925 11:03:56.928128 8173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:56.927451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:50733)
0925 11:03:56.927874 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:56.927967 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":212,"threads_started":1}
I20260925 11:03:57.364529 8174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:57.363855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:50533)
0925 11:03:57.364284 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:57.364372 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":219,"threads_started":1}
I20260925 11:03:57.508765 8175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:57.507977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:44619)
0925 11:03:57.508468 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:57.508554 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":220,"threads_started":1}
I20260925 11:03:57.929750 8176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:57.928988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:46209)
0925 11:03:57.929449 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:57.929532 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":203,"threads_started":1}
I20260925 11:03:58.366393 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:58.365480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:40251)
0925 11:03:58.366050 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:58.366139 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":244,"threads_started":1}
I20260925 11:03:58.510330 8178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:58.509597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:50893)
0925 11:03:58.510039 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:58.510128 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":213,"threads_started":1}
I20260925 11:03:58.931234 8179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:58.930569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:53627)
0925 11:03:58.930990 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:58.931068 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":213,"threads_started":1}
I20260925 11:03:59.367923 8180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:59.367230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:47311)
0925 11:03:59.367680 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:59.367766 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":220,"threads_started":1}
I20260925 11:03:59.511942 8181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:59.511203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:36491)
0925 11:03:59.511668 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:59.511779 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":215,"threads_started":1}
I20260925 11:03:59.932895 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:03:59.932141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:35751)
0925 11:03:59.932617 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0925 11:03:59.932716 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":216,"threads_started":1}
I20260925 11:04:00.369536 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:00.368752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:56583)
0925 11:04:00.369256 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:00.369366 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":268,"threads_started":1}
I20260925 11:04:00.513541 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:00.512846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:39601)
0925 11:04:00.513296 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:00.513381 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":234,"threads_started":1}
I20260925 11:04:00.934554 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:00.933816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:54039)
0925 11:04:00.934315 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:00.934395 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":264,"threads_started":1}
I20260925 11:04:01.371169 8186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:01.370428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:36603)
0925 11:04:01.370914 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:01.371002 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":279,"threads_started":1}
I20260925 11:04:01.515153 8187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:01.514429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:33305)
0925 11:04:01.514851 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:01.514956 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":200,"threads_started":1}
I20260925 11:04:01.936146 8188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:01.935404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:41205)
0925 11:04:01.935855 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:01.935950 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":214,"threads_started":1}
I20260925 11:04:02.372699 8189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:02.371994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:56243)
0925 11:04:02.372426 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:02.372518 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":213,"threads_started":1}
I20260925 11:04:02.516674 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:02.516007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:57883)
0925 11:04:02.516426 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:02.516507 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":215,"threads_started":1}
I20260925 11:04:02.937688 8191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:02.937009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:54375)
0925 11:04:02.937429 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:02.937511 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":213,"threads_started":1}
I20260925 11:04:03.374418 8192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:03.373642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:38625)
0925 11:04:03.374134 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:03.374258 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":236,"threads_started":1}
I20260925 11:04:03.518476 8193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:03.517615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:45489)
0925 11:04:03.518141 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:03.518285 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":240,"threads_started":1}
I20260925 11:04:03.939230 8194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:03.938496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:37035)
0925 11:04:03.938933 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:03.939042 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":214,"threads_started":1}
I20260925 11:04:04.376066 8195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:04.375311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:59545)
0925 11:04:04.375782 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:04.375880 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":223,"threads_started":1}
I20260925 11:04:04.520183 8196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:04.519411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:41673)
0925 11:04:04.519906 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:04.519997 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":243,"threads_started":1}
I20260925 11:04:04.940745 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:04.940070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:46953)
0925 11:04:04.940474 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:04.940553 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":199,"threads_started":1}
I20260925 11:04:05.377712 8198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:05.376963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:42595)
0925 11:04:05.377436 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:05.377526 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":254,"threads_started":1}
I20260925 11:04:05.522117 8199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:05.521365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.193:54965)
0925 11:04:05.521826 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:05.521929 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
I20260925 11:04:05.942539 8200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:05.941797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.194:37261)
0925 11:04:05.942219 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:05.942325 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":202,"threads_started":1}
I20260925 11:04:06.364251 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 7725
I20260925 11:04:06.379493 8202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:06.378807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:44035 (local address 127.24.159.195:38087)
0925 11:04:06.379222 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:06.379317 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:44035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":210,"threads_started":1}
I20260925 11:04:06.382818 8201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:06.365386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49390 (local address 127.24.159.193:33653)
0925 11:04:06.366001 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:06.366019 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:06.366080 (+ 61us) server_negotiation.cc:407] Connection header received
0925 11:04:06.368189 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:06.368200 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:06.368631 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:06.368937 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:06.370281 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:06.371497 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:06.373133 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:06.374004 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:06.374167 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:06.376653 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:06.376661 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:06.376677 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:06.376729 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:06.379847 (+ 3118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:06.380452 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:06.380458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:06.380464 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:06.380623 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:06.380962 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:06.380966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:06.380968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:06.381501 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:06.381734 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:06.382093 (+ 359us) server_negotiation.cc:299] Negotiation successful
0925 11:04:06.382539 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":273,"threads_started":1}
I20260925 11:04:06.384280 7884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:06.465863 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 7725
I20260925 11:04:06.480028 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 7892
I20260925 11:04:06.496402 8204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:06.480983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.24.159.194:39897)
0925 11:04:06.481544 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:06.481559 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:06.481597 (+ 38us) server_negotiation.cc:407] Connection header received
0925 11:04:06.483263 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:06.483272 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:06.483706 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:06.484034 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:06.485273 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:06.486539 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:06.487747 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:06.488391 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:06.488515 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:06.490994 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:06.491001 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:06.491016 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:06.491058 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:06.493562 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:06.494157 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:06.494163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:06.494169 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:06.494425 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:06.494739 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:06.494742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:06.494744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:06.495255 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:06.495486 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:06.495796 (+ 310us) server_negotiation.cc:299] Negotiation successful
0925 11:04:06.496124 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":255,"threads_started":1}
I20260925 11:04:06.497764 8016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:06.578724 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 7892
I20260925 11:04:06.592873 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 8024
I20260925 11:04:06.609797 8206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:06.593935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56512 (local address 127.24.159.195:37427)
0925 11:04:06.594661 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:06.594679 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:06.594728 (+ 49us) server_negotiation.cc:407] Connection header received
0925 11:04:06.596329 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:06.596340 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:06.596787 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:06.597068 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:06.598479 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:06.599713 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:06.600990 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:06.601691 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:06.601821 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:06.604302 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:06.604310 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:06.604326 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:06.604373 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:06.607069 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:06.607696 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:06.607701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:06.607708 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:06.607863 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:06.608142 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:06.608145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:06.608147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:06.608648 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:06.608877 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:06.609182 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 11:04:06.609506 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":362,"threads_started":1}
I20260925 11:04:06.611145 8148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:06.692010 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 8024
2026-09-25T11:04:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (23890 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:04:06 dist-test-slave-trms krb5kdc[8210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:04:06 dist-test-slave-trms krb5kdc[8210](info): set up 2 sockets
Sep 25 11:04:06 dist-test-slave-trms krb5kdc[8210](info): commencing operation
krb5kdc: starting...
W20260925 11:04:07.694638 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.968s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:04:07 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:04:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:04:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:04:07.921068 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36321
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:36321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:04:08.240749 8223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:08.241079 8223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:08.241214 8223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:08.248383 8223 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 11:04:08.248489 8223 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 11:04:08.248543 8223 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 11:04:08.248615 8223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:04:08.248682 8223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:08.248777 8223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:08.248834 8223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:04:08.248904 8223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:04:08.259248 8223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:36321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:08.260888 8223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:08.262622 8223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:08.274775 8231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:08.274926 8228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:08.274753 8229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:08.275799 8223 server_base.cc:1034] running on GCE node
I20260925 11:04:08.276523 8223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:08.277930 8223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:08.279145 8223 hybrid_clock.cc:648] HybridClock initialized: now 1790334248279069 us; error 51 us; skew 500 ppm
I20260925 11:04:08.279449 8223 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:08 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334248, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:08.282473 8223 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:04:08.283978 8223 webserver.cc:460] Webserver started at http://127.24.159.254:35233/ using document root <none> and password file <none>
I20260925 11:04:08.284653 8223 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:08.284739 8223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:08.284999 8223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:04:08.288045 8223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "4dfd4f139783497983159ef299f8862a"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "35a53a08153ac3344839371c7e244746"
server_key_iv: "a186468ad714ba9da0e4b7478db8b2ee"
server_key_version: "encryptionkey@0"
I20260925 11:04:08.288784 8223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "4dfd4f139783497983159ef299f8862a"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "35a53a08153ac3344839371c7e244746"
server_key_iv: "a186468ad714ba9da0e4b7478db8b2ee"
server_key_version: "encryptionkey@0"
I20260925 11:04:08.293846 8223 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260925 11:04:08.297197 8238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:08.298524 8223 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 11:04:08.298655 8223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "4dfd4f139783497983159ef299f8862a"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "35a53a08153ac3344839371c7e244746"
server_key_iv: "a186468ad714ba9da0e4b7478db8b2ee"
server_key_version: "encryptionkey@0"
I20260925 11:04:08.298825 8223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:08.320508 8223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:08.324476 8223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:08.324690 8223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:08.341470 8223 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:36321
I20260925 11:04:08.341534 8290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:36321 every 8 connection(s)
I20260925 11:04:08.342775 8223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:04:08.346426 8291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:04:08.352049 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 8223
I20260925 11:04:08.352272 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:04:08.353091 25215 external_mini_cluster.cc:1428] Setting key 1f8f10223f10e91e62131d36540e6d6c
I20260925 11:04:08.357631 8291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Bootstrap starting.
I20260925 11:04:08.361285 8291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Neither blocks nor log segments found. Creating new log.
I20260925 11:04:08.362587 8291 log.cc:824] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:08.365847 8291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: No bootstrap required, opened a new log
Sep 25 11:04:08 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:04:08.374142 8291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:08.374545 8291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:04:08.374652 8291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dfd4f139783497983159ef299f8862a, State: Initialized, Role: FOLLOWER
I20260925 11:04:08.375332 8291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:08.375574 8291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:04:08.375699 8291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:04:08.375864 8291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:04:08.376262 8294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:08.355213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33892 (local address 127.24.159.254:36321)
0925 11:04:08.355988 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:08.355999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:08.356039 (+ 40us) server_negotiation.cc:407] Connection header received
0925 11:04:08.357668 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:08.357690 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:08.358113 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:08.358481 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:08.360253 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:08.361583 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:08.363492 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:08.364319 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:08.364449 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:08.369014 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:08.369021 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:08.369037 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:08.369078 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:08.372203 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:08.372887 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:08.372897 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:08.372904 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:08.373079 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:08.373459 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:08.373462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:08.373464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:08.374011 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:08.374314 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:08.374763 (+ 449us) server_negotiation.cc:299] Negotiation successful
0925 11:04:08.375148 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":358,"threads_started":1}
I20260925 11:04:08.378266 8291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:08.378764 8291 leader_election.cc:302] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dfd4f139783497983159ef299f8862a; no voters:
I20260925 11:04:08.379366 8291 leader_election.cc:288] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:04:08.379456 8296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:04:08.380046 8296 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 1 LEADER]: Becoming Leader. State: Replica: 4dfd4f139783497983159ef299f8862a, State: Running, Role: LEADER
I20260925 11:04:08.380589 8296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:08.381517 8291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:04:08.383394 8297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4dfd4f139783497983159ef299f8862a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } } }
I20260925 11:04:08.383813 8297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: This master's current role is: LEADER
I20260925 11:04:08.383395 8298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dfd4f139783497983159ef299f8862a. Latest consensus state: current_term: 1 leader_uuid: "4dfd4f139783497983159ef299f8862a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } } }
I20260925 11:04:08.386739 8298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: This master's current role is: LEADER
I20260925 11:04:08.391440 8302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:04:08.395819 8302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:04:08.403937 8302 catalog_manager.cc:1347] Generated new cluster ID: 1afaa20f8d9d4624ac2623a25a70813d
I20260925 11:04:08.404034 8302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:04:08.413538 8302 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:04:08.414877 8302 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:04:08.420965 8302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Generated new TSK 0
I20260925 11:04:08.421747 8302 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:04:08.487303 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36321
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:04:08.815578 8317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:08.815850 8317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:08.815968 8317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:08.823117 8317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:08.823225 8317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:08.823371 8317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:04:08.832978 8317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:08.834244 8317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:08.835525 8317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:08.847925 8323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:08.847965 8325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:08.848182 8322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:08.848616 8317 server_base.cc:1034] running on GCE node
I20260925 11:04:08.849197 8317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:08.850090 8317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:08.851279 8317 hybrid_clock.cc:648] HybridClock initialized: now 1790334248851226 us; error 43 us; skew 500 ppm
I20260925 11:04:08.851552 8317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:08 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334248, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:08.854525 8317 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:04:08.855813 8317 webserver.cc:460] Webserver started at http://127.24.159.193:35171/ using document root <none> and password file <none>
I20260925 11:04:08.856416 8317 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:08.856508 8317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:08.856757 8317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:04:08.859537 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "0177b285cbf44f1d9a77f65e15555b59"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "efdae6a3d56cb43158273eadd173aff9"
server_key_iv: "dce3df2274c95aea157abcd8de535511"
server_key_version: "encryptionkey@0"
I20260925 11:04:08.860235 8317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "0177b285cbf44f1d9a77f65e15555b59"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "efdae6a3d56cb43158273eadd173aff9"
server_key_iv: "dce3df2274c95aea157abcd8de535511"
server_key_version: "encryptionkey@0"
I20260925 11:04:08.865093 8317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:04:08.868296 8332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:08.869518 8317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 11:04:08.869653 8317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "0177b285cbf44f1d9a77f65e15555b59"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "efdae6a3d56cb43158273eadd173aff9"
server_key_iv: "dce3df2274c95aea157abcd8de535511"
server_key_version: "encryptionkey@0"
I20260925 11:04:08.869804 8317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:08.882789 8317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:08.886799 8317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:08.887030 8317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:08.887980 8317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:04:08.889259 8317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:04:08.889335 8317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:08.889447 8317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:04:08.889503 8317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:08.918829 8317 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:45269
I20260925 11:04:08.918879 8444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:45269 every 8 connection(s)
I20260925 11:04:08.920066 8317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:04:08.922235 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 8317
I20260925 11:04:08.922433 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:04:08.923188 25215 external_mini_cluster.cc:1428] Setting key c5f0cc89ff469e1b720d1487fb5985d3
Sep 25 11:04:08 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334248, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:04:08.941195 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:08.922580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:49295 (local address 127.24.159.254:36321)
0925 11:04:08.923024 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:08.923031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:08.924047 (+ 1016us) server_negotiation.cc:407] Connection header received
0925 11:04:08.926297 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:08.926301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:08.926362 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:08.926448 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:08.928381 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:08.929085 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:08.931251 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:08.931638 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:08.931727 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:08.935700 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:08.935705 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:08.935708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:08.935741 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:08.938145 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:08.938904 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:08.938908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:08.938910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:08.939030 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:08.939461 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:08.939465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:08.939467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:08.939750 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:08.939883 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:08.940605 (+ 722us) server_negotiation.cc:299] Negotiation successful
0925 11:04:08.940841 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":196,"threads_started":1}
I20260925 11:04:08.942219 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:08.923162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:49295)
0925 11:04:08.923901 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:08.923945 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:08.926018 (+ 2073us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:08.926719 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:08.926730 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:08.927474 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:04:08.928230 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:08.928253 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:08.929240 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:08.929244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:08.931129 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:08.931143 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:08.931847 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:08.931853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:08.932115 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:08.932799 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:04:08.932826 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:04:08.935468 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:04:08.938306 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:08.938316 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:08.938335 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:08.938775 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:08.939133 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:08.939138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:08.939142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:08.939346 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:08.939874 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:04:08.939883 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:04:08.940243 (+ 360us) client_negotiation.cc:769] Sending connection context
0925 11:04:08.940478 (+ 235us) client_negotiation.cc:240] Negotiation successful
0925 11:04:08.941018 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":303,"threads_started":1}
I20260925 11:04:08.943976 8445 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36321
I20260925 11:04:08.944283 8445 heartbeater.cc:464] Registering TS with master...
I20260925 11:04:08.944988 8445 heartbeater.cc:511] Master 127.24.159.254:36321 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
I20260925 11:04:08.947295 8255 ts_manager.cc:194] Registered new tserver with Master: 0177b285cbf44f1d9a77f65e15555b59 (127.24.159.193:45269)
I20260925 11:04:08.948755 8255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:49295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:04:08.973567 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36321
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:04:09.282541 8451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:09.282853 8451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:09.283001 8451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:09.290361 8451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:09.290478 8451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:09.290659 8451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:04:09.300490 8451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:09.301805 8451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:09.303182 8451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:09.316138 8457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:09.316179 8456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:09.316200 8459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:09.316710 8451 server_base.cc:1034] running on GCE node
I20260925 11:04:09.317400 8451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:09.318339 8451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:09.319533 8451 hybrid_clock.cc:648] HybridClock initialized: now 1790334249319471 us; error 57 us; skew 500 ppm
I20260925 11:04:09.319841 8451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:09 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334249, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:09.329288 8451 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:04:09.330776 8451 webserver.cc:460] Webserver started at http://127.24.159.194:40633/ using document root <none> and password file <none>
I20260925 11:04:09.331455 8451 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:09.331557 8451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:09.331808 8451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:04:09.334858 8451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "333f9069efa62ab79212374ef95afe9b"
server_key_iv: "cc64cd70ed719a2870faecb075e3bb84"
server_key_version: "encryptionkey@0"
I20260925 11:04:09.335598 8451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "333f9069efa62ab79212374ef95afe9b"
server_key_iv: "cc64cd70ed719a2870faecb075e3bb84"
server_key_version: "encryptionkey@0"
I20260925 11:04:09.340807 8451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:04:09.345139 8466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:09.346712 8451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 11:04:09.346865 8451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "333f9069efa62ab79212374ef95afe9b"
server_key_iv: "cc64cd70ed719a2870faecb075e3bb84"
server_key_version: "encryptionkey@0"
I20260925 11:04:09.347062 8451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:09.377864 8451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:09.382241 8451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:09.382488 8451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:09.383399 8451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:04:09.384686 8451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:04:09.384758 8451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:09.384871 8451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:04:09.384919 8451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:09.414242 8451 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:42729
I20260925 11:04:09.414275 8580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:42729 every 8 connection(s)
I20260925 11:04:09.415488 8451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:04:09.423418 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 8451
I20260925 11:04:09.423593 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:04:09.424019 25215 external_mini_cluster.cc:1428] Setting key 1915ba43c58c009db8381d64d370d4b1
Sep 25 11:04:09 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334249, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:04:09.435626 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.417881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:33801 (local address 127.24.159.254:36321)
0925 11:04:09.418050 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:09.418055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:09.419186 (+ 1131us) server_negotiation.cc:407] Connection header received
0925 11:04:09.421012 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:09.421018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:09.421100 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:09.421196 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:09.422823 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.423547 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.425246 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.425698 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.425839 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.429642 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:09.429648 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:09.429652 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:09.429691 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:09.432605 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:09.433321 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:09.433326 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:09.433330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:09.433488 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:09.433860 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:09.433865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:09.433867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:09.434253 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:09.434397 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:09.435197 (+ 800us) server_negotiation.cc:299] Negotiation successful
0925 11:04:09.435376 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:04:09.437024 8583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.418323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:33801)
0925 11:04:09.419010 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:09.419054 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:09.420740 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:09.421452 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:09.421460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:09.422016 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:04:09.422646 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.422664 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.423707 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.423711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.425104 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.425116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.425933 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.425941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.426267 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.426940 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:04:09.426968 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:04:09.429415 (+ 2447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:04:09.432751 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:09.432758 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:09.432772 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:09.433204 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:09.433581 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:09.433584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:09.433586 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:09.433764 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:09.434365 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:04:09.434374 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:04:09.434818 (+ 444us) client_negotiation.cc:769] Sending connection context
0925 11:04:09.435113 (+ 295us) client_negotiation.cc:240] Negotiation successful
0925 11:04:09.435677 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":262,"threads_started":1}
I20260925 11:04:09.438478 8581 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36321
I20260925 11:04:09.438791 8581 heartbeater.cc:464] Registering TS with master...
I20260925 11:04:09.439587 8581 heartbeater.cc:511] Master 127.24.159.254:36321 requested a full tablet report, sending...
I20260925 11:04:09.441045 8255 ts_manager.cc:194] Registered new tserver with Master: 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729)
I20260925 11:04:09.441813 8255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:33801
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:04:09.474943 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36321
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:04:09.785915 8586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:09.786237 8586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:09.786373 8586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:09.793751 8586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:09.793884 8586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:09.794088 8586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:04:09.804126 8586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:09.805445 8586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:09.806790 8586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:09.819490 8591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:09.819468 8594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:09.819614 8592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:09.820212 8586 server_base.cc:1034] running on GCE node
I20260925 11:04:09.820814 8586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:09.821715 8586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:09.822908 8586 hybrid_clock.cc:648] HybridClock initialized: now 1790334249822859 us; error 49 us; skew 500 ppm
I20260925 11:04:09.823239 8586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:09 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334249, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:09.826135 8586 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:04:09.827530 8586 webserver.cc:460] Webserver started at http://127.24.159.195:36873/ using document root <none> and password file <none>
I20260925 11:04:09.828192 8586 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:09.828307 8586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:09.828604 8586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:04:09.831538 8586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "f6be516d2cc06adcc748b52cab6b2abe"
server_key_iv: "9a7c1658caded301a76907b9478cc736"
server_key_version: "encryptionkey@0"
I20260925 11:04:09.832273 8586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "f6be516d2cc06adcc748b52cab6b2abe"
server_key_iv: "9a7c1658caded301a76907b9478cc736"
server_key_version: "encryptionkey@0"
I20260925 11:04:09.837296 8586 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:04:09.840745 8601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:09.842060 8586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:04:09.842252 8586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "f6be516d2cc06adcc748b52cab6b2abe"
server_key_iv: "9a7c1658caded301a76907b9478cc736"
server_key_version: "encryptionkey@0"
I20260925 11:04:09.842444 8586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:09.861516 8586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:09.865279 8586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:09.865540 8586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:09.866493 8586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:04:09.867762 8586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:04:09.867837 8586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:09.867959 8586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:04:09.868016 8586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:09.897570 8586 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:46031
I20260925 11:04:09.897617 8715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:46031 every 8 connection(s)
I20260925 11:04:09.898854 8586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:04:09.904146 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 8586
I20260925 11:04:09.904310 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:04:09.904785 25215 external_mini_cluster.cc:1428] Setting key dc947b4706ea40f6ed629f0681410094
Sep 25 11:04:09 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334249, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:04:09.917611 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.901264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:56313 (local address 127.24.159.254:36321)
0925 11:04:09.901475 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:09.901481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:09.902668 (+ 1187us) server_negotiation.cc:407] Connection header received
0925 11:04:09.904612 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:09.904618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:09.904692 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:09.904779 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:09.906502 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.907352 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.908915 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.909346 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.909449 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.912737 (+ 3288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:09.912742 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:09.912745 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:09.912779 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:09.915068 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:09.915709 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:09.915712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:09.915715 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:09.915834 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:09.916163 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:09.916165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:09.916167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:09.916428 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:09.916531 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:09.917191 (+ 660us) server_negotiation.cc:299] Negotiation successful
0925 11:04:09.917356 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 11:04:09.918848 8718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.901754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:56313)
0925 11:04:09.902530 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:09.902569 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:09.904275 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:09.905026 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:09.905036 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:09.905651 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:04:09.906325 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.906351 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.907497 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.907502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.908779 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.908793 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.909523 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.909528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.909747 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.910422 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:04:09.910451 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:04:09.912541 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:04:09.915188 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:09.915195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:09.915210 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:09.915617 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:09.915910 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:09.915913 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:09.915915 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:09.916085 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:09.916514 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:04:09.916519 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:04:09.916850 (+ 331us) client_negotiation.cc:769] Sending connection context
0925 11:04:09.917126 (+ 276us) client_negotiation.cc:240] Negotiation successful
0925 11:04:09.917535 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":266,"threads_started":1}
I20260925 11:04:09.923074 8716 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36321
I20260925 11:04:09.923364 8716 heartbeater.cc:464] Registering TS with master...
I20260925 11:04:09.924077 8716 heartbeater.cc:511] Master 127.24.159.254:36321 requested a full tablet report, sending...
I20260925 11:04:09.925451 8255 ts_manager.cc:194] Registered new tserver with Master: 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031)
I20260925 11:04:09.926306 8255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:56313
I20260925 11:04:09.932008 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:04:09.947853 8447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.936210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38712 (local address 127.24.159.254:36321)
0925 11:04:09.936356 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:09.936360 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:09.936862 (+ 502us) server_negotiation.cc:407] Connection header received
0925 11:04:09.936959 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:09.936963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:09.937026 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:09.937106 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:09.938589 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.939313 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.940510 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.940895 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.941011 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.943177 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:09.943182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:09.943186 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:09.943216 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:09.945630 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:09.946256 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:09.946260 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:09.946262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:09.946397 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:09.946736 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:09.946739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:09.946741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:09.947032 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:09.947237 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:09.947471 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 11:04:09.947628 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 11:04:09.951720 8445 heartbeater.cc:503] Master 127.24.159.254:36321 was elected leader, sending a full tablet report...
I20260925 11:04:09.951541 8255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38712:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:04:09.958585 8255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:04:09.987149 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.969660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.24.159.194:42729)
0925 11:04:09.977313 (+ 7653us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:09.977324 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:09.977365 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:04:09.977488 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:09.977497 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:09.977811 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:09.978031 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:09.979022 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.980706 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.984269 (+ 3563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.985741 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.985895 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.986082 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:09.986743 (+ 661us) server_negotiation.cc:299] Negotiation successful
0925 11:04:09.986924 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":248,"threads_started":1}
I20260925 11:04:09.987619 8728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.969475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:42729 (local address 127.0.0.1:51466)
0925 11:04:09.970605 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:09.970647 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:09.970803 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:09.978081 (+ 7278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:09.978090 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:09.978129 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:09.978601 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.978616 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.980901 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.980907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.983515 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.983533 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.985928 (+ 2395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.985934 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.986143 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.986219 (+ 76us) client_negotiation.cc:769] Sending connection context
0925 11:04:09.986329 (+ 110us) client_negotiation.cc:240] Negotiation successful
0925 11:04:09.986510 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":905,"thread_start_us":786,"threads_started":1}
I20260925 11:04:09.991245 8730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.969815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:46031 (local address 127.0.0.1:55572)
0925 11:04:09.983044 (+ 13229us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:09.983066 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:09.983180 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:09.984505 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:09.984509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:09.984530 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:09.984849 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.984858 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.986679 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.986684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.988741 (+ 2057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.988757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.990477 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.990483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.990645 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.990679 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 11:04:09.990743 (+ 64us) client_negotiation.cc:240] Negotiation successful
0925 11:04:09.990863 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13082,"mutex_wait_us":1,"thread_start_us":12944,"threads_started":1}
I20260925 11:04:09.991641 8514 tablet_service.cc:1467] Processing CreateTablet for tablet 5d958f03c0204e608c799532dbb11f78 (DEFAULT_TABLE table=test-table [id=2896382bc76248aa92bcee171ce4a96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:04:09.992332 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.970430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55572 (local address 127.24.159.195:46031)
0925 11:04:09.983883 (+ 13453us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:09.983891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:09.983928 (+ 37us) server_negotiation.cc:407] Connection header received
0925 11:04:09.984034 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:09.984043 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:09.984307 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:09.984500 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:09.985011 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.986541 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.988896 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.990258 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.991621 (+ 1363us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.991792 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:09.991854 (+ 62us) server_negotiation.cc:299] Negotiation successful
0925 11:04:09.992002 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13274,"thread_start_us":14511,"threads_started":1}
I20260925 11:04:09.993134 8514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d958f03c0204e608c799532dbb11f78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:04:09.993239 8732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.971055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.0.0.1:49566)
0925 11:04:09.984252 (+ 13197us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:09.984270 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:09.984372 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:09.986609 (+ 2237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:09.986615 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:09.986641 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:09.987016 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.987030 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.989063 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.989068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.990939 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:09.990955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.992381 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.992389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:09.992661 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.992704 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 11:04:09.992800 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 11:04:09.992915 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13083,"thread_start_us":12969,"threads_started":1}
I20260925 11:04:09.993324 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:09.971370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.24.159.193:45269)
0925 11:04:09.985647 (+ 14277us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:09.985654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:09.985682 (+ 28us) server_negotiation.cc:407] Connection header received
0925 11:04:09.985771 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:09.985778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:09.986400 (+ 622us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:09.987229 (+ 829us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:09.987460 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.988947 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.991082 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:09.992129 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:09.992321 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:09.992473 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:09.992857 (+ 384us) server_negotiation.cc:299] Negotiation successful
0925 11:04:09.993006 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14112,"thread_start_us":13949,"threads_started":1}
I20260925 11:04:09.995986 8650 tablet_service.cc:1467] Processing CreateTablet for tablet 5d958f03c0204e608c799532dbb11f78 (DEFAULT_TABLE table=test-table [id=2896382bc76248aa92bcee171ce4a96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:04:09.997478 8650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d958f03c0204e608c799532dbb11f78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:04:09.997776 8380 tablet_service.cc:1467] Processing CreateTablet for tablet 5d958f03c0204e608c799532dbb11f78 (DEFAULT_TABLE table=test-table [id=2896382bc76248aa92bcee171ce4a96b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:04:09.999832 8380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d958f03c0204e608c799532dbb11f78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:04:10.005250 8734 tablet_bootstrap.cc:492] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: Bootstrap starting.
I20260925 11:04:10.008108 8734 tablet_bootstrap.cc:654] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: Neither blocks nor log segments found. Creating new log.
I20260925 11:04:10.009111 8734 log.cc:824] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:10.012421 8734 tablet_bootstrap.cc:492] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: No bootstrap required, opened a new log
I20260925 11:04:10.012751 8734 ts_tablet_manager.cc:1397] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260925 11:04:10.014043 8736 tablet_bootstrap.cc:492] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Bootstrap starting.
I20260925 11:04:10.016047 8737 tablet_bootstrap.cc:492] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: Bootstrap starting.
I20260925 11:04:10.017596 8736 tablet_bootstrap.cc:654] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Neither blocks nor log segments found. Creating new log.
I20260925 11:04:10.018855 8736 log.cc:824] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:10.018954 8737 tablet_bootstrap.cc:654] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: Neither blocks nor log segments found. Creating new log.
I20260925 11:04:10.020093 8737 log.cc:824] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:10.022090 8736 tablet_bootstrap.cc:492] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: No bootstrap required, opened a new log
I20260925 11:04:10.022390 8736 ts_tablet_manager.cc:1397] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260925 11:04:10.022537 8734 raft_consensus.cc:348] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.023002 8734 raft_consensus.cc:374] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:04:10.023034 8737 tablet_bootstrap.cc:492] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: No bootstrap required, opened a new log
I20260925 11:04:10.023118 8734 raft_consensus.cc:729] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7131beabbe6f4b559cc1f3ef7c98fcd9, State: Initialized, Role: FOLLOWER
I20260925 11:04:10.023294 8737 ts_tablet_manager.cc:1397] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260925 11:04:10.023864 8734 consensus_queue.cc:260] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.024982 8581 heartbeater.cc:503] Master 127.24.159.254:36321 was elected leader, sending a full tablet report...
I20260925 11:04:10.025625 8734 ts_tablet_manager.cc:1428] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260925 11:04:10.031023 8737 raft_consensus.cc:348] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.031533 8737 raft_consensus.cc:374] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:04:10.031687 8737 raft_consensus.cc:729] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0177b285cbf44f1d9a77f65e15555b59, State: Initialized, Role: FOLLOWER
I20260925 11:04:10.031986 8736 raft_consensus.cc:348] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.032459 8736 raft_consensus.cc:374] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:04:10.032586 8736 raft_consensus.cc:729] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f19df8f16ff4e2d94ed1a6fbd5b718b, State: Initialized, Role: FOLLOWER
I20260925 11:04:10.032480 8737 consensus_queue.cc:260] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.033183 8736 consensus_queue.cc:260] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.037967 8716 heartbeater.cc:503] Master 127.24.159.254:36321 was elected leader, sending a full tablet report...
I20260925 11:04:10.038558 8736 ts_tablet_manager.cc:1428] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260925 11:04:10.039795 8737 ts_tablet_manager.cc:1428] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: Time spent starting tablet: real 0.016s user 0.006s sys 0.010s
W20260925 11:04:10.150408 8717 tablet.cc:2369] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:04:10.167331 8582 tablet.cc:2369] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:04:10.172252 8446 tablet.cc:2369] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:04:10.366777 8740 raft_consensus.cc:482] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:04:10.367131 8740 raft_consensus.cc:504] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.368768 8740 leader_election.cc:288] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031), 0177b285cbf44f1d9a77f65e15555b59 (127.24.159.193:45269)
I20260925 11:04:10.373035 8742 raft_consensus.cc:482] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:04:10.373320 8742 raft_consensus.cc:504] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.375247 8742 leader_election.cc:288] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729), 0177b285cbf44f1d9a77f65e15555b59 (127.24.159.193:45269)
I20260925 11:04:10.377632 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.369215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:43565 (local address 127.24.159.193:45269)
0925 11:04:10.369414 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:10.369421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:10.370070 (+ 649us) server_negotiation.cc:407] Connection header received
0925 11:04:10.370307 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:10.370313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:10.370391 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:10.370486 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:10.371246 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.372329 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.375395 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.376573 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.376691 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.376757 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:10.377274 (+ 517us) server_negotiation.cc:299] Negotiation successful
0925 11:04:10.377392 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:04:10.377768 8744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.369041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.24.159.194:43565)
0925 11:04:10.369949 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:10.369969 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:10.370139 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:10.370547 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:10.370552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:10.370644 (+ 92us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:10.371091 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:10.371100 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.372456 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.372460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:10.375213 (+ 2753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:10.375229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.376945 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.376952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:10.377122 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.377149 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 11:04:10.377225 (+ 76us) client_negotiation.cc:240] Negotiation successful
0925 11:04:10.377352 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":35,"spinlock_wait_cycles":24832,"thread_start_us":596,"threads_started":1}
I20260925 11:04:10.378832 8400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d958f03c0204e608c799532dbb11f78" candidate_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0177b285cbf44f1d9a77f65e15555b59" is_pre_election: true
I20260925 11:04:10.379457 8400 raft_consensus.cc:2454] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7131beabbe6f4b559cc1f3ef7c98fcd9 in term 0.
I20260925 11:04:10.380317 8469 leader_election.cc:302] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59, 7131beabbe6f4b559cc1f3ef7c98fcd9; no voters:
I20260925 11:04:10.380740 8740 raft_consensus.cc:2781] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:04:10.380872 8740 raft_consensus.cc:482] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:04:10.380971 8740 raft_consensus.cc:3037] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:04:10.382800 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.369287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:52625 (local address 127.24.159.195:46031)
0925 11:04:10.369446 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:10.369452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:10.370137 (+ 685us) server_negotiation.cc:407] Connection header received
0925 11:04:10.370307 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:10.370313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:10.370390 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:10.370486 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:10.371248 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.372348 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.374885 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.376158 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.376301 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.376372 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:10.382409 (+ 6037us) server_negotiation.cc:299] Negotiation successful
0925 11:04:10.382545 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 11:04:10.383443 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.375987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:47687 (local address 127.24.159.194:42729)
0925 11:04:10.376148 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:10.376153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:10.376171 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:04:10.376225 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:10.376229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:10.376299 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:10.376380 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:10.377003 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.378208 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.380717 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.382166 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.382326 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.382388 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:10.383022 (+ 634us) server_negotiation.cc:299] Negotiation successful
0925 11:04:10.383166 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":52}
I20260925 11:04:10.383590 8743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.369186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:46031 (local address 127.24.159.194:52625)
0925 11:04:10.370039 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:10.370053 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:10.370142 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:10.370547 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:10.370552 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:10.370584 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:10.371080 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:10.371094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.372453 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.372458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:10.374702 (+ 2244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:10.374722 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.381544 (+ 6822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.381550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:10.381752 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.381792 (+ 40us) client_negotiation.cc:769] Sending connection context
0925 11:04:10.383022 (+ 1230us) client_negotiation.cc:240] Negotiation successful
0925 11:04:10.383216 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"mutex_wait_us":49,"thread_start_us":432,"threads_started":1}
I20260925 11:04:10.383864 8718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.375413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:42729 (local address 127.24.159.195:47687)
0925 11:04:10.375584 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:10.375601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:10.375717 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:10.376530 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:10.376535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:10.376564 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:10.376832 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:10.376844 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.378383 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.378390 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:10.380491 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:10.380512 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.382459 (+ 1947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.382465 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:10.382660 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.382703 (+ 43us) client_negotiation.cc:769] Sending connection context
0925 11:04:10.382790 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 11:04:10.382910 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 11:04:10.384500 8670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d958f03c0204e608c799532dbb11f78" candidate_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" is_pre_election: true
I20260925 11:04:10.384951 8670 raft_consensus.cc:2454] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7131beabbe6f4b559cc1f3ef7c98fcd9 in term 0.
I20260925 11:04:10.385038 8535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d958f03c0204e608c799532dbb11f78" candidate_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" is_pre_election: true
I20260925 11:04:10.385766 8740 raft_consensus.cc:504] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.385970 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.376060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:54893 (local address 127.24.159.193:45269)
0925 11:04:10.378117 (+ 2057us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:10.378122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:10.378136 (+ 14us) server_negotiation.cc:407] Connection header received
0925 11:04:10.378208 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:10.378213 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:10.378275 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:10.378351 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:10.379127 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.380185 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.382458 (+ 2273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.383620 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.384248 (+ 628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.384315 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:10.385639 (+ 1324us) server_negotiation.cc:299] Negotiation successful
0925 11:04:10.385773 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1962,"thread_start_us":8200,"threads_started":1}
I20260925 11:04:10.386031 8745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.375749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.24.159.195:54893)
0925 11:04:10.376794 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:10.376811 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:10.376910 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:10.378496 (+ 1586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:10.378501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:10.378528 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:10.378926 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:10.378942 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.380359 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.380364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:10.382295 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:10.382309 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.384494 (+ 2185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.384501 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:10.385466 (+ 965us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.385520 (+ 54us) client_negotiation.cc:769] Sending connection context
0925 11:04:10.385614 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 11:04:10.385734 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":737,"threads_started":1}
I20260925 11:04:10.386209 8535 raft_consensus.cc:2379] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7f19df8f16ff4e2d94ed1a6fbd5b718b in current term 1: Already voted for candidate 7131beabbe6f4b559cc1f3ef7c98fcd9 in this term.
I20260925 11:04:10.386682 8400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d958f03c0204e608c799532dbb11f78" candidate_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0177b285cbf44f1d9a77f65e15555b59" is_pre_election: true
I20260925 11:04:10.386895 8400 raft_consensus.cc:2454] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f19df8f16ff4e2d94ed1a6fbd5b718b in term 0.
I20260925 11:04:10.387238 8604 leader_election.cc:302] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59, 7f19df8f16ff4e2d94ed1a6fbd5b718b; no voters: 7131beabbe6f4b559cc1f3ef7c98fcd9
I20260925 11:04:10.387442 8670 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d958f03c0204e608c799532dbb11f78" candidate_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b"
I20260925 11:04:10.387627 8670 raft_consensus.cc:3037] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:04:10.387846 8740 leader_election.cc:288] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [CANDIDATE]: Term 1 election: Requested vote from peers 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031), 0177b285cbf44f1d9a77f65e15555b59 (127.24.159.193:45269)
I20260925 11:04:10.387832 8400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d958f03c0204e608c799532dbb11f78" candidate_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0177b285cbf44f1d9a77f65e15555b59"
I20260925 11:04:10.388018 8400 raft_consensus.cc:3037] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:04:10.390976 8400 raft_consensus.cc:2454] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7131beabbe6f4b559cc1f3ef7c98fcd9 in term 1.
I20260925 11:04:10.390976 8670 raft_consensus.cc:2454] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7131beabbe6f4b559cc1f3ef7c98fcd9 in term 1.
I20260925 11:04:10.391234 8742 raft_consensus.cc:2741] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260925 11:04:10.391420 8469 leader_election.cc:302] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59, 7131beabbe6f4b559cc1f3ef7c98fcd9; no voters:
I20260925 11:04:10.391731 8740 raft_consensus.cc:2781] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:04:10.392278 8740 raft_consensus.cc:686] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 LEADER]: Becoming Leader. State: Replica: 7131beabbe6f4b559cc1f3ef7c98fcd9, State: Running, Role: LEADER
I20260925 11:04:10.392735 8740 consensus_queue.cc:237] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } }
I20260925 11:04:10.397089 8255 catalog_manager.cc:5495] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194). New cstate: current_term: 1 leader_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } health_report { overall_health: UNKNOWN } } }
I20260925 11:04:10.450424 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.445272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.24.159.194:42729)
0925 11:04:10.445450 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:10.445455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:10.445471 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:04:10.445557 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:10.445561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:10.445646 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:10.445778 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:04:10.446147 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.446957 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.448530 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.448941 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.449031 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.449439 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:04:10.449950 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:04:10.450057 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:10.450097 (+ 40us) server_negotiation.cc:299] Negotiation successful
0925 11:04:10.450162 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 11:04:10.458602 8670 raft_consensus.cc:1261] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 1 FOLLOWER]: Refusing update from remote peer 7131beabbe6f4b559cc1f3ef7c98fcd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:04:10.458602 8400 raft_consensus.cc:1261] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Refusing update from remote peer 7131beabbe6f4b559cc1f3ef7c98fcd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:04:10.459513 8747 consensus_queue.cc:1035] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:04:10.459824 8740 consensus_queue.cc:1035] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:04:10.483065 8254 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:04:10.483345 8254 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38712:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:04:10.486864 8254 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Sending DeleteTablet for 3 replicas of tablet 5d958f03c0204e608c799532dbb11f78
I20260925 11:04:10.487958 8514 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d958f03c0204e608c799532dbb11f78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:04:10 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:51466
I20260925 11:04:10.488036 8650 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d958f03c0204e608c799532dbb11f78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:04:10 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:55572
I20260925 11:04:10.488885 8760 tablet_replica.cc:330] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: stopping tablet replica
I20260925 11:04:10.489063 8380 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d958f03c0204e608c799532dbb11f78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:04:10 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:49566
I20260925 11:04:10.489378 8760 raft_consensus.cc:2229] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:04:10.490083 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 8317
I20260925 11:04:10.490000 8760 raft_consensus.cc:2258] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:04:10.490970 8759 tablet_replica.cc:330] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: stopping tablet replica
I20260925 11:04:10.491470 8759 raft_consensus.cc:2229] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:04:10.491982 8759 raft_consensus.cc:2258] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:04:10.494117 8759 ts_tablet_manager.cc:1905] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:04:10.495553 8760 ts_tablet_manager.cc:1905] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:04:10.500747 8761 tablet_replica.cc:330] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: stopping tablet replica
I20260925 11:04:10.501030 8759 ts_tablet_manager.cc:1918] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:04:10.501268 8759 log.cc:1196] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/5d958f03c0204e608c799532dbb11f78
I20260925 11:04:10.501211 8761 raft_consensus.cc:2229] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:04:10.501745 8761 raft_consensus.cc:2258] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:04:10.501921 8759 ts_tablet_manager.cc:1939] T 5d958f03c0204e608c799532dbb11f78 P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Deleting consensus metadata
I20260925 11:04:10.503962 8242 catalog_manager.cc:4841] TS 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031): tablet 5d958f03c0204e608c799532dbb11f78 (table test-table [id=2896382bc76248aa92bcee171ce4a96b]) successfully deleted
I20260925 11:04:10.505563 8761 ts_tablet_manager.cc:1905] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:04:10.505887 8760 ts_tablet_manager.cc:1918] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:04:10.506040 8760 log.cc:1196] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/5d958f03c0204e608c799532dbb11f78
I20260925 11:04:10.506577 8760 ts_tablet_manager.cc:1939] T 5d958f03c0204e608c799532dbb11f78 P 7131beabbe6f4b559cc1f3ef7c98fcd9: Deleting consensus metadata
I20260925 11:04:10.508234 8241 catalog_manager.cc:4841] TS 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729): tablet 5d958f03c0204e608c799532dbb11f78 (table test-table [id=2896382bc76248aa92bcee171ce4a96b]) successfully deleted
I20260925 11:04:10.511729 8761 ts_tablet_manager.cc:1918] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:04:10.511868 8761 log.cc:1196] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/5d958f03c0204e608c799532dbb11f78
I20260925 11:04:10.512562 8761 ts_tablet_manager.cc:1939] T 5d958f03c0204e608c799532dbb11f78 P 0177b285cbf44f1d9a77f65e15555b59: Deleting consensus metadata
I20260925 11:04:10.514384 8241 catalog_manager.cc:4841] TS 0177b285cbf44f1d9a77f65e15555b59 (127.24.159.193:45269): tablet 5d958f03c0204e608c799532dbb11f78 (table test-table [id=2896382bc76248aa92bcee171ce4a96b]) successfully deleted
Sep 25 11:04:10 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:04:10.522887 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.491563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49576 (local address 127.24.159.193:45269)
0925 11:04:10.491724 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:10.491730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:10.506833 (+ 15103us) server_negotiation.cc:407] Connection header received
0925 11:04:10.507146 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:10.507153 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:10.507275 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:10.507361 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:10.509518 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.510679 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.512693 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.513283 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.513407 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.517492 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:10.517503 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:10.517519 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:10.517559 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:10.520269 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:10.520924 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:10.520931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:10.520933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:10.521058 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:10.521413 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:10.521417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:10.521419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:10.521752 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:10.522018 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:10.522369 (+ 351us) server_negotiation.cc:299] Negotiation successful
0925 11:04:10.522594 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:04:10.523501 8440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:10.619453 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 8317
I20260925 11:04:10.632961 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 8451
Sep 25 11:04:10 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:04:10.650442 8729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.633712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51482 (local address 127.24.159.194:42729)
0925 11:04:10.633918 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:10.633922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:10.634074 (+ 152us) server_negotiation.cc:407] Connection header received
0925 11:04:10.634371 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:10.634378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:10.634453 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:10.634537 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:10.636314 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.637051 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.638930 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.639336 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.639429 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.644456 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:10.644471 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:10.644488 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:10.644530 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:10.647736 (+ 3206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:10.648508 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:10.648516 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:10.648518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:10.648653 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:10.649023 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:10.649026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:10.649028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:10.649404 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:10.649667 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:10.649941 (+ 274us) server_negotiation.cc:299] Negotiation successful
0925 11:04:10.650123 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:04:10.651160 8576 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:10.763475 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 8451
I20260925 11:04:10.778505 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 8586
Sep 25 11:04:10 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:04:10.793169 8731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:10.779263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55582 (local address 127.24.159.195:46031)
0925 11:04:10.779443 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:10.779449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:10.779468 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:04:10.779635 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:10.779641 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:10.779724 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:10.779803 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:10.781101 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.781827 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.783121 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:10.783524 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:10.783630 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:10.787794 (+ 4164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:10.787811 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:10.787826 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:10.787862 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:10.790380 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:10.791105 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:10.791112 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:10.791115 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:10.791265 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:10.791615 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:10.791620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:10.791624 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:10.792010 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:10.792288 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:10.792612 (+ 324us) server_negotiation.cc:299] Negotiation successful
0925 11:04:10.792826 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:04:10.793793 8711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:10.881633 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 8586
I20260925 11:04:10.897344 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 8223
I20260925 11:04:10.898490 8286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:11.006901 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 8223
I20260925 11:04:31.024766 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36321
--webserver_interface=127.24.159.254
--webserver_port=35233
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:36321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:04:31.342970 8767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:31.343273 8767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:31.343410 8767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:31.350493 8767 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 11:04:31.350592 8767 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 11:04:31.350663 8767 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 11:04:31.350733 8767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:04:31.350806 8767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:31.350899 8767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:31.350960 8767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:04:31.351033 8767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:04:31.360801 8767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:36321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=35233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:31.362110 8767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:31.363451 8767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:31.384934 8773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:31.384882 8776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:31.384913 8774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:31.385222 8767 server_base.cc:1034] running on GCE node
I20260925 11:04:31.386358 8767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:31.387918 8767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:31.389146 8767 hybrid_clock.cc:648] HybridClock initialized: now 1790334271389077 us; error 39 us; skew 500 ppm
I20260925 11:04:31.389528 8767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:31 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334271, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:31.396256 8767 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:04:31.397938 8767 webserver.cc:460] Webserver started at http://127.24.159.254:35233/ using document root <none> and password file <none>
I20260925 11:04:31.398804 8767 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:31.398929 8767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:31.405293 8767 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260925 11:04:31.408339 8783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:31.409737 8767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:04:31.409906 8767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "4dfd4f139783497983159ef299f8862a"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "35a53a08153ac3344839371c7e244746"
server_key_iv: "a186468ad714ba9da0e4b7478db8b2ee"
server_key_version: "encryptionkey@0"
I20260925 11:04:31.411000 8767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:31.430739 8767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:31.435384 8767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:31.435662 8767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:31.453100 8767 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:36321
I20260925 11:04:31.453120 8835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:36321 every 8 connection(s)
I20260925 11:04:31.454643 8767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:04:31.457237 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 8767
I20260925 11:04:31.458140 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:45269
--local_ip_for_outbound_sockets=127.24.159.193
--tserver_master_addrs=127.24.159.254:36321
--webserver_port=35171
--webserver_interface=127.24.159.193
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:04:31.463631 8836 sys_catalog.cc:263] Verifying existing consensus state
I20260925 11:04:31.467895 8836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Bootstrap starting.
I20260925 11:04:31.487720 8836 log.cc:824] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:31.499708 8836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 11:04:31.500214 8836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Bootstrap complete.
I20260925 11:04:31.508890 8836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:31.509748 8836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dfd4f139783497983159ef299f8862a, State: Initialized, Role: FOLLOWER
I20260925 11:04:31.510385 8836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:31.510607 8836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:04:31.510711 8836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:04:31.510865 8836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 1 FOLLOWER]: Advancing to term 2
I20260925 11:04:31.514609 8836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:31.515134 8836 leader_election.cc:302] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dfd4f139783497983159ef299f8862a; no voters:
I20260925 11:04:31.515765 8836 leader_election.cc:288] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260925 11:04:31.515933 8840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 2 FOLLOWER]: Leader election won for term 2
I20260925 11:04:31.517620 8836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:04:31.518457 8840 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 2 LEADER]: Becoming Leader. State: Replica: 4dfd4f139783497983159ef299f8862a, State: Running, Role: LEADER
I20260925 11:04:31.519129 8840 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:31.528141 8841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4dfd4f139783497983159ef299f8862a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } } }
I20260925 11:04:31.528566 8841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: This master's current role is: LEADER
I20260925 11:04:31.529198 8842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dfd4f139783497983159ef299f8862a. Latest consensus state: current_term: 2 leader_uuid: "4dfd4f139783497983159ef299f8862a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } } }
I20260925 11:04:31.529501 8842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: This master's current role is: LEADER
I20260925 11:04:31.537458 8854 catalog_manager.cc:1259] Loaded cluster ID: 1afaa20f8d9d4624ac2623a25a70813d
I20260925 11:04:31.537575 8854 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: loading cluster ID for follower catalog manager: success
I20260925 11:04:31.541468 8854 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: acquiring CA information for follower catalog manager: success
I20260925 11:04:31.544301 8854 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 11:04:31.544549 8856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:04:31.548519 8856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:04:31.549752 8856 catalog_manager.cc:1259] Loaded cluster ID: 1afaa20f8d9d4624ac2623a25a70813d
I20260925 11:04:31.549849 8856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:04:31.551576 8856 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:04:31.552769 8856 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Loaded TSK: 0
I20260925 11:04:31.585180 8856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Generated new TSK 1
I20260925 11:04:31.585596 8856 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 11:04:31.794095 8838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:31.794445 8838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:31.794589 8838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:31.802251 8838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:31.802382 8838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:31.802537 8838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:04:31.812704 8838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:45269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=35171
--tserver_master_addrs=127.24.159.254:36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:31.814043 8838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:31.815492 8838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:31.829933 8863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:31.829960 8864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:31.830669 8866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:31.831266 8838 server_base.cc:1034] running on GCE node
I20260925 11:04:31.831877 8838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:31.832845 8838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:31.834057 8838 hybrid_clock.cc:648] HybridClock initialized: now 1790334271834000 us; error 86 us; skew 500 ppm
I20260925 11:04:31.834443 8838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:31 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334271, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:31.837399 8838 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:04:31.838928 8838 webserver.cc:460] Webserver started at http://127.24.159.193:35171/ using document root <none> and password file <none>
I20260925 11:04:31.839598 8838 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:31.839713 8838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:31.844995 8838 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260925 11:04:31.847692 8873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:31.849107 8838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:04:31.849242 8838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "0177b285cbf44f1d9a77f65e15555b59"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "efdae6a3d56cb43158273eadd173aff9"
server_key_iv: "dce3df2274c95aea157abcd8de535511"
server_key_version: "encryptionkey@0"
I20260925 11:04:31.850157 8838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:31.863592 8838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:31.867827 8838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:31.868090 8838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:31.869143 8838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:04:31.870570 8838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:04:31.870666 8838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:31.870769 8838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:04:31.870841 8838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:31.902082 8838 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:45269
I20260925 11:04:31.902155 8986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:45269 every 8 connection(s)
I20260925 11:04:31.903601 8838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:04:31.913884 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 8838
I20260925 11:04:31.914577 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:42729
--local_ip_for_outbound_sockets=127.24.159.194
--tserver_master_addrs=127.24.159.254:36321
--webserver_port=40633
--webserver_interface=127.24.159.194
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:04:31 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334271, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:04:31.927002 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:31.906481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:44323 (local address 127.24.159.254:36321)
0925 11:04:31.907252 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:31.907269 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:31.907679 (+ 410us) server_negotiation.cc:407] Connection header received
0925 11:04:31.909721 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:31.909733 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:31.910086 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:31.910447 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:31.911961 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:31.913222 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:31.914917 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:31.915643 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:31.915825 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:31.919531 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:31.919539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:31.919556 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:31.919600 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:31.922541 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:31.923283 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:31.923290 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:31.923296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:31.923456 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:31.923864 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:31.923867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:31.923870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:31.924345 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:31.924520 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:31.925305 (+ 785us) server_negotiation.cc:299] Negotiation successful
0925 11:04:31.925692 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":326,"threads_started":1}
I20260925 11:04:31.927002 8990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:31.906677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:44323)
0925 11:04:31.907515 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:31.907558 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:31.909215 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:31.910621 (+ 1406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:31.910634 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:31.911235 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:04:31.911779 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:31.911796 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:31.913392 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:31.913396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:31.914786 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:31.914802 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:31.915864 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:31.915871 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:31.916142 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:31.916807 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:04:31.916830 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:04:31.919250 (+ 2420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:04:31.922677 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:31.922687 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:31.922703 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:31.923149 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:31.923562 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:31.923567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:31.923572 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:31.923751 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:31.924474 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:04:31.924482 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:04:31.924818 (+ 336us) client_negotiation.cc:769] Sending connection context
0925 11:04:31.925049 (+ 231us) client_negotiation.cc:240] Negotiation successful
0925 11:04:31.925626 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":305,"threads_started":1}
I20260925 11:04:31.930790 8987 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36321
I20260925 11:04:31.931108 8987 heartbeater.cc:464] Registering TS with master...
I20260925 11:04:31.931852 8987 heartbeater.cc:511] Master 127.24.159.254:36321 requested a full tablet report, sending...
I20260925 11:04:31.934449 8800 ts_manager.cc:194] Registered new tserver with Master: 0177b285cbf44f1d9a77f65e15555b59 (127.24.159.193:45269)
I20260925 11:04:31.936769 8800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:44323
W20260925 11:04:32.239669 8991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:32.239948 8991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:32.240053 8991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:32.247269 8991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:32.247381 8991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:32.247502 8991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:04:32.257507 8991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:42729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=40633
--tserver_master_addrs=127.24.159.254:36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:32.258829 8991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:32.260154 8991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:32.272753 8999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:32.272773 8997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:32.273111 8996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:32.273558 8991 server_base.cc:1034] running on GCE node
I20260925 11:04:32.274267 8991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:32.275269 8991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:32.276536 8991 hybrid_clock.cc:648] HybridClock initialized: now 1790334272276479 us; error 39 us; skew 500 ppm
I20260925 11:04:32.276877 8991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:32 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334272, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:32.279969 8991 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:04:32.281414 8991 webserver.cc:460] Webserver started at http://127.24.159.194:40633/ using document root <none> and password file <none>
I20260925 11:04:32.282083 8991 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:32.282229 8991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:32.287577 8991 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260925 11:04:32.290339 9006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:32.291680 8991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:04:32.291822 8991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "333f9069efa62ab79212374ef95afe9b"
server_key_iv: "cc64cd70ed719a2870faecb075e3bb84"
server_key_version: "encryptionkey@0"
I20260925 11:04:32.292804 8991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:32.314361 8991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:32.318806 8991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:32.319039 8991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:32.320031 8991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:04:32.321448 8991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:04:32.321534 8991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:32.321640 8991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:04:32.321727 8991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:32.352095 8991 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:42729
I20260925 11:04:32.352175 9118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:42729 every 8 connection(s)
I20260925 11:04:32.353538 8991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:04:32.359349 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 8991
I20260925 11:04:32.360046 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:46031
--local_ip_for_outbound_sockets=127.24.159.195
--tserver_master_addrs=127.24.159.254:36321
--webserver_port=36873
--webserver_interface=127.24.159.195
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 25 11:04:32 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334272, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:04:32.375352 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.356139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:51597 (local address 127.24.159.254:36321)
0925 11:04:32.356302 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:32.356307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:32.357479 (+ 1172us) server_negotiation.cc:407] Connection header received
0925 11:04:32.359581 (+ 2102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:32.359586 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:32.359665 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:32.359767 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:32.361870 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.362626 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.364710 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.365152 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.365261 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.369367 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:32.369373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:32.369376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:32.369411 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:32.372379 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:32.373100 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:32.373105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:32.373108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:32.373248 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:32.373619 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:32.373623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:32.373626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:32.373942 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:32.374089 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:32.374775 (+ 686us) server_negotiation.cc:299] Negotiation successful
0925 11:04:32.375014 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 11:04:32.376235 9122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.356606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:51597)
0925 11:04:32.357305 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:32.357349 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:32.359188 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:32.360074 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:32.360087 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:32.360840 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:04:32.361664 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.361692 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.362785 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.362791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.364541 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.364555 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.365389 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.365396 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.365678 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.366434 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:04:32.366471 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:04:32.369118 (+ 2647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:04:32.372521 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:32.372529 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:32.372545 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:32.372986 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:32.373339 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:32.373342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:32.373344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:32.373525 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:32.374042 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:04:32.374048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:04:32.374433 (+ 385us) client_negotiation.cc:769] Sending connection context
0925 11:04:32.374677 (+ 244us) client_negotiation.cc:240] Negotiation successful
0925 11:04:32.375082 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":264,"threads_started":1}
I20260925 11:04:32.377597 9119 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36321
I20260925 11:04:32.377888 9119 heartbeater.cc:464] Registering TS with master...
I20260925 11:04:32.378629 9119 heartbeater.cc:511] Master 127.24.159.254:36321 requested a full tablet report, sending...
I20260925 11:04:32.380059 8800 ts_manager.cc:194] Registered new tserver with Master: 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729)
I20260925 11:04:32.380877 8800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:51597
W20260925 11:04:32.686115 9123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:32.686494 9123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:32.686645 9123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:32.693943 9123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:32.694069 9123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:32.694272 9123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:04:32.704624 9123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:46031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=36873
--tserver_master_addrs=127.24.159.254:36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:32.705996 9123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:32.707412 9123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:32.721127 9129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:32.721165 9128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:32.721066 9131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:32.723583 9123 server_base.cc:1034] running on GCE node
I20260925 11:04:32.724195 9123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:32.725165 9123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:32.726365 9123 hybrid_clock.cc:648] HybridClock initialized: now 1790334272726312 us; error 45 us; skew 500 ppm
I20260925 11:04:32.726691 9123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:32 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334272, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:32.729836 9123 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:04:32.731369 9123 webserver.cc:460] Webserver started at http://127.24.159.195:36873/ using document root <none> and password file <none>
I20260925 11:04:32.732070 9123 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:32.732182 9123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:32.737855 9123 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260925 11:04:32.740720 9138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:32.742094 9123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 11:04:32.742293 9123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "f6be516d2cc06adcc748b52cab6b2abe"
server_key_iv: "9a7c1658caded301a76907b9478cc736"
server_key_version: "encryptionkey@0"
I20260925 11:04:32.743309 9123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:32.759481 9123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:32.763463 9123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:32.763718 9123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:32.764741 9123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:04:32.766124 9123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:04:32.766237 9123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:32.766333 9123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:04:32.766389 9123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:32.796300 9123 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:46031
I20260925 11:04:32.796379 9250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:46031 every 8 connection(s)
I20260925 11:04:32.797807 9123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:04:32.806545 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 9123
Sep 25 11:04:32 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334272, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:04:32.818337 8989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.800295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:55199 (local address 127.24.159.254:36321)
0925 11:04:32.800481 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:32.800486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:32.801681 (+ 1195us) server_negotiation.cc:407] Connection header received
0925 11:04:32.803534 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:32.803540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:32.803624 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:32.803710 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:32.805228 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.805956 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.807745 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.808350 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.808435 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.812109 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:32.812115 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:32.812120 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:32.812161 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:32.814796 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:32.815734 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:32.815738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:32.815741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:32.815877 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:32.816318 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:32.816323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:32.816327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:32.816679 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:32.816816 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:32.817552 (+ 736us) server_negotiation.cc:299] Negotiation successful
0925 11:04:32.817846 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":64}
I20260925 11:04:32.819097 9253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.800799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:55199)
0925 11:04:32.801524 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:32.801563 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:32.803265 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:32.803960 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:32.803969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:32.804491 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:04:32.805060 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.805078 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.806128 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.806132 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.807563 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.807579 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.808662 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.808670 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.808883 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.809490 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:04:32.809512 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:04:32.811899 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:04:32.814940 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:32.814950 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:32.814972 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:32.815627 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:32.815965 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:32.815968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:32.815970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:32.816191 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:32.816794 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:04:32.816802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:04:32.817177 (+ 375us) client_negotiation.cc:769] Sending connection context
0925 11:04:32.817415 (+ 238us) client_negotiation.cc:240] Negotiation successful
0925 11:04:32.817878 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":278,"threads_started":1}
I20260925 11:04:32.820559 9251 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36321
I20260925 11:04:32.820860 9251 heartbeater.cc:464] Registering TS with master...
I20260925 11:04:32.821604 9251 heartbeater.cc:511] Master 127.24.159.254:36321 requested a full tablet report, sending...
I20260925 11:04:32.821817 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.807433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.24.159.254:36321)
0925 11:04:32.808225 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:32.808229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:32.808244 (+ 15us) server_negotiation.cc:407] Connection header received
0925 11:04:32.808283 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:32.808287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:32.808340 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:32.808422 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:32.809974 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.810991 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.812477 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.812892 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.813000 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.816046 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:32.816051 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:32.816054 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:32.816080 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:32.818970 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:32.819788 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:32.819793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:32.819796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:32.819930 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:32.820318 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:32.820322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:32.820325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:32.820658 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:32.820959 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:32.821235 (+ 276us) server_negotiation.cc:299] Negotiation successful
0925 11:04:32.821437 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":692,"thread_start_us":538,"threads_started":1}
I20260925 11:04:32.823225 8800 ts_manager.cc:194] Registered new tserver with Master: 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031)
I20260925 11:04:32.824124 8800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:55199
I20260925 11:04:32.824324 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:04:32.830493 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.825194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.24.159.254:36321)
0925 11:04:32.825366 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:32.825372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:32.825809 (+ 437us) server_negotiation.cc:407] Connection header received
0925 11:04:32.825981 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:32.825986 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:32.826156 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:32.826351 (+ 195us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:04:32.826735 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.827520 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.828919 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.829320 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.829405 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.829796 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:04:32.829983 (+ 187us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0925 11:04:32.830294 (+ 311us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35696: token expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260925 11:04:32.830598 9255 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35696: token expired
W20260925 11:04:32.831130 8721 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.159.254:36321)
I20260925 11:04:32.843039 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.831510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35702 (local address 127.24.159.254:36321)
0925 11:04:32.831646 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:32.831650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:32.831686 (+ 36us) server_negotiation.cc:407] Connection header received
0925 11:04:32.831800 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:32.831803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:32.831853 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:32.831920 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:32.833073 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.833770 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.835189 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.835578 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.835691 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.838223 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:32.838229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:32.838234 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:32.838273 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:32.840575 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:32.841188 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:32.841194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:32.841198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:32.841331 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:32.841656 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:32.841661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:32.841665 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:32.841967 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:32.842231 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:32.842511 (+ 280us) server_negotiation.cc:299] Negotiation successful
0925 11:04:32.842725 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260925 11:04:32.846355 8800 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:04:32.853758 8800 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:04:32.875515 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.864836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.0.0.1:33738)
0925 11:04:32.865578 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:32.865632 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:32.865820 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:32.867115 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:32.867123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:32.867155 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:32.867597 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.867609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.869832 (+ 2223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.869839 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.872390 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.872408 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.874480 (+ 2072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.874487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.874724 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.874787 (+ 63us) client_negotiation.cc:769] Sending connection context
0925 11:04:32.874909 (+ 122us) client_negotiation.cc:240] Negotiation successful
0925 11:04:32.875103 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":484,"threads_started":1}
I20260925 11:04:32.876866 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.864955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33738 (local address 127.24.159.193:45269)
0925 11:04:32.866437 (+ 1482us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:32.866446 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:32.866473 (+ 27us) server_negotiation.cc:407] Connection header received
0925 11:04:32.866570 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:32.866580 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:32.866871 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:32.867799 (+ 928us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:32.868008 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.869663 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.872641 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.874210 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.876085 (+ 1875us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.876288 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:32.876359 (+ 71us) server_negotiation.cc:299] Negotiation successful
0925 11:04:32.876539 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321,"thread_start_us":1408,"threads_started":1}
I20260925 11:04:32.883016 8921 tablet_service.cc:1467] Processing CreateTablet for tablet 748a635605b242259cd5363e49e4c16f (DEFAULT_TABLE table=test-table [id=f8f9c12e2af3424a8d4659051fc42a27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:04:32.884308 8921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748a635605b242259cd5363e49e4c16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:04:32.885413 9261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.866146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:42729 (local address 127.0.0.1:43800)
0925 11:04:32.875781 (+ 9635us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:32.875800 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:32.875895 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:32.878716 (+ 2821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:32.878724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:32.878766 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:32.878910 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.878920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.880918 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.880923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.883053 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.883068 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.884707 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.884712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.884830 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.884852 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 11:04:32.884907 (+ 55us) client_negotiation.cc:240] Negotiation successful
0925 11:04:32.885043 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9542,"mutex_wait_us":80,"spinlock_wait_cycles":1408,"thread_start_us":12190,"threads_started":1}
I20260925 11:04:32.885442 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.865472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:46031 (local address 127.0.0.1:35686)
0925 11:04:32.874248 (+ 8776us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:32.874270 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:32.874405 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:32.878550 (+ 4145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:32.878554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:32.878580 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:32.878990 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.879002 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.880952 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.880956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.882840 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:32.882855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.884599 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.884606 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:32.884769 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.884803 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 11:04:32.884899 (+ 96us) client_negotiation.cc:240] Negotiation successful
0925 11:04:32.885031 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8627,"mutex_wait_us":47,"thread_start_us":8485,"threads_started":1}
I20260925 11:04:32.886324 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.866593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.24.159.194:42729)
0925 11:04:32.878145 (+ 11552us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:32.878153 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:32.878182 (+ 29us) server_negotiation.cc:407] Connection header received
0925 11:04:32.878292 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:32.878299 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:32.878511 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:32.879048 (+ 537us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:32.879255 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.880775 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.883207 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.884534 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.885646 (+ 1112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.885808 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:32.885872 (+ 64us) server_negotiation.cc:299] Negotiation successful
0925 11:04:32.886015 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11394,"thread_start_us":11309,"threads_started":1}
I20260925 11:04:32.886555 9265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:32.866064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35686 (local address 127.24.159.195:46031)
0925 11:04:32.877868 (+ 11804us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:32.877877 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:32.877913 (+ 36us) server_negotiation.cc:407] Connection header received
0925 11:04:32.878015 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:32.878023 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:32.878322 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:32.878512 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:32.879201 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.880783 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.882976 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:32.884284 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:32.884465 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:32.884649 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:32.884954 (+ 305us) server_negotiation.cc:299] Negotiation successful
0925 11:04:32.886229 (+ 1275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11640,"thread_start_us":11475,"threads_started":1}
I20260925 11:04:32.889648 9186 tablet_service.cc:1467] Processing CreateTablet for tablet 748a635605b242259cd5363e49e4c16f (DEFAULT_TABLE table=test-table [id=f8f9c12e2af3424a8d4659051fc42a27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:04:32.890384 9054 tablet_service.cc:1467] Processing CreateTablet for tablet 748a635605b242259cd5363e49e4c16f (DEFAULT_TABLE table=test-table [id=f8f9c12e2af3424a8d4659051fc42a27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:04:32.890919 9186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748a635605b242259cd5363e49e4c16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:04:32.891698 9054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 748a635605b242259cd5363e49e4c16f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:04:32.899335 9267 tablet_bootstrap.cc:492] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Bootstrap starting.
I20260925 11:04:32.903416 9267 tablet_bootstrap.cc:654] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Neither blocks nor log segments found. Creating new log.
I20260925 11:04:32.904974 9267 log.cc:824] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:32.909205 9269 tablet_bootstrap.cc:492] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: Bootstrap starting.
I20260925 11:04:32.912839 9269 tablet_bootstrap.cc:654] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: Neither blocks nor log segments found. Creating new log.
I20260925 11:04:32.916262 9268 tablet_bootstrap.cc:492] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Bootstrap starting.
I20260925 11:04:32.918497 9269 log.cc:824] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:32.920054 9268 tablet_bootstrap.cc:654] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Neither blocks nor log segments found. Creating new log.
I20260925 11:04:32.921072 9269 tablet_bootstrap.cc:492] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: No bootstrap required, opened a new log
I20260925 11:04:32.921120 9267 tablet_bootstrap.cc:492] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: No bootstrap required, opened a new log
I20260925 11:04:32.921348 9269 ts_tablet_manager.cc:1397] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260925 11:04:32.921401 9267 ts_tablet_manager.cc:1397] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.011s
I20260925 11:04:32.921365 9268 log.cc:824] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:32.924597 9268 tablet_bootstrap.cc:492] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: No bootstrap required, opened a new log
I20260925 11:04:32.924883 9268 ts_tablet_manager.cc:1397] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260925 11:04:32.929097 9269 raft_consensus.cc:348] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:32.929482 9269 raft_consensus.cc:374] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:04:32.929569 9269 raft_consensus.cc:729] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7131beabbe6f4b559cc1f3ef7c98fcd9, State: Initialized, Role: FOLLOWER
I20260925 11:04:32.930157 9269 consensus_queue.cc:260] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:32.931248 9119 heartbeater.cc:503] Master 127.24.159.254:36321 was elected leader, sending a full tablet report...
I20260925 11:04:32.931584 9267 raft_consensus.cc:348] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:32.932045 9267 raft_consensus.cc:374] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:04:32.932152 9267 raft_consensus.cc:729] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0177b285cbf44f1d9a77f65e15555b59, State: Initialized, Role: FOLLOWER
I20260925 11:04:32.932790 9267 consensus_queue.cc:260] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:32.933753 9269 ts_tablet_manager.cc:1428] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260925 11:04:32.934876 8987 heartbeater.cc:503] Master 127.24.159.254:36321 was elected leader, sending a full tablet report...
I20260925 11:04:32.935530 9267 ts_tablet_manager.cc:1428] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Time spent starting tablet: real 0.014s user 0.008s sys 0.008s
I20260925 11:04:32.936661 9268 raft_consensus.cc:348] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:32.937005 9268 raft_consensus.cc:374] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:04:32.937146 9268 raft_consensus.cc:729] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f19df8f16ff4e2d94ed1a6fbd5b718b, State: Initialized, Role: FOLLOWER
I20260925 11:04:32.937737 9268 consensus_queue.cc:260] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:32.938731 9251 heartbeater.cc:503] Master 127.24.159.254:36321 was elected leader, sending a full tablet report...
I20260925 11:04:32.939344 9268 ts_tablet_manager.cc:1428] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260925 11:04:33.042755 9273 raft_consensus.cc:482] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:04:33.043033 9273 raft_consensus.cc:504] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:33.044623 9273 leader_election.cc:288] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0177b285cbf44f1d9a77f65e15555b59 (127.24.159.193:45269), 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031)
W20260925 11:04:33.049299 9252 tablet.cc:2369] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:04:33.053498 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.045150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:38521 (local address 127.24.159.193:45269)
0925 11:04:33.045350 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:33.045356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:33.046029 (+ 673us) server_negotiation.cc:407] Connection header received
0925 11:04:33.046364 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:33.046369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:33.046437 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:33.046515 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:33.047262 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.048396 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.051145 (+ 2749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.052336 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.052489 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:33.052562 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:33.053095 (+ 533us) server_negotiation.cc:299] Negotiation successful
0925 11:04:33.053236 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:04:33.053498 9265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.045163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:35265 (local address 127.24.159.195:46031)
0925 11:04:33.045349 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:33.045356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:33.046046 (+ 690us) server_negotiation.cc:407] Connection header received
0925 11:04:33.046330 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:33.046336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:33.046414 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:33.046510 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:04:33.047309 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.048361 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.051129 (+ 2768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.052336 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.052490 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:33.052562 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:33.053090 (+ 528us) server_negotiation.cc:299] Negotiation successful
0925 11:04:33.053263 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:04:33.053785 9276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.045002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:46031 (local address 127.24.159.194:35265)
0925 11:04:33.045948 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:33.045962 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:33.046086 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:33.046534 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:33.046539 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:33.046585 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:33.047170 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:33.047180 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.048495 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.048500 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:33.050975 (+ 2475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:33.050992 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.052599 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.052607 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:33.052854 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:33.052963 (+ 109us) client_negotiation.cc:769] Sending connection context
0925 11:04:33.053063 (+ 100us) client_negotiation.cc:240] Negotiation successful
0925 11:04:33.053254 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":833,"mutex_wait_us":200,"spinlock_wait_cycles":14592,"thread_start_us":550,"threads_started":1}
I20260925 11:04:33.053792 9277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.044873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.24.159.194:38521)
0925 11:04:33.045893 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:33.045914 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:33.046123 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:33.046540 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:33.046543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:33.046587 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:04:33.047109 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:33.047127 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.048498 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.048503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:33.050976 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:33.050992 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.052604 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.052609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:33.052856 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:33.052963 (+ 107us) client_negotiation.cc:769] Sending connection context
0925 11:04:33.053056 (+ 93us) client_negotiation.cc:240] Negotiation successful
0925 11:04:33.053254 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":270,"spinlock_wait_cycles":49792,"thread_start_us":697,"threads_started":1}
I20260925 11:04:33.054991 9206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748a635605b242259cd5363e49e4c16f" candidate_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" is_pre_election: true
I20260925 11:04:33.054991 8941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748a635605b242259cd5363e49e4c16f" candidate_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0177b285cbf44f1d9a77f65e15555b59" is_pre_election: true
I20260925 11:04:33.055565 8941 raft_consensus.cc:2454] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7131beabbe6f4b559cc1f3ef7c98fcd9 in term 0.
I20260925 11:04:33.055565 9206 raft_consensus.cc:2454] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7131beabbe6f4b559cc1f3ef7c98fcd9 in term 0.
I20260925 11:04:33.056288 9009 leader_election.cc:302] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59, 7131beabbe6f4b559cc1f3ef7c98fcd9; no voters:
I20260925 11:04:33.056711 9273 raft_consensus.cc:2781] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:04:33.056842 9273 raft_consensus.cc:482] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:04:33.056938 9273 raft_consensus.cc:3037] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:04:33.059188 9273 raft_consensus.cc:504] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:33.059773 9273 leader_election.cc:288] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [CANDIDATE]: Term 1 election: Requested vote from peers 0177b285cbf44f1d9a77f65e15555b59 (127.24.159.193:45269), 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031)
I20260925 11:04:33.060211 8941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748a635605b242259cd5363e49e4c16f" candidate_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0177b285cbf44f1d9a77f65e15555b59"
I20260925 11:04:33.060230 9206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "748a635605b242259cd5363e49e4c16f" candidate_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b"
I20260925 11:04:33.060392 8941 raft_consensus.cc:3037] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:04:33.060400 9206 raft_consensus.cc:3037] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:04:33.062565 9206 raft_consensus.cc:2454] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7131beabbe6f4b559cc1f3ef7c98fcd9 in term 1.
I20260925 11:04:33.062582 8941 raft_consensus.cc:2454] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7131beabbe6f4b559cc1f3ef7c98fcd9 in term 1.
I20260925 11:04:33.062963 9009 leader_election.cc:302] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59, 7131beabbe6f4b559cc1f3ef7c98fcd9; no voters:
I20260925 11:04:33.063241 9273 raft_consensus.cc:2781] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:04:33.063642 9273 raft_consensus.cc:686] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 LEADER]: Becoming Leader. State: Replica: 7131beabbe6f4b559cc1f3ef7c98fcd9, State: Running, Role: LEADER
I20260925 11:04:33.064136 9273 consensus_queue.cc:237] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:33.068697 8799 catalog_manager.cc:5495] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194). New cstate: current_term: 1 leader_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } health_report { overall_health: UNKNOWN } } }
W20260925 11:04:33.105568 9121 tablet.cc:2369] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:04:33.125355 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.120144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.24.159.194:42729)
0925 11:04:33.120342 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:33.120348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:33.120365 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:04:33.120544 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:33.120548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:33.120630 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:33.120756 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:04:33.121127 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.121879 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.123275 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.123668 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.123771 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:33.124254 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:04:33.124757 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:04:33.124887 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:33.125003 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 11:04:33.125083 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:04:33.133051 8941 raft_consensus.cc:1261] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Refusing update from remote peer 7131beabbe6f4b559cc1f3ef7c98fcd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:04:33.133064 9206 raft_consensus.cc:1261] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 1 FOLLOWER]: Refusing update from remote peer 7131beabbe6f4b559cc1f3ef7c98fcd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:04:33.133981 9278 consensus_queue.cc:1035] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:04:33.134311 9273 consensus_queue.cc:1035] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:04:33.144650 9280 mvcc.cc:205] Tried to move back new op lower bound from 7333209182744010752 to 7333209182474653696. Current Snapshot: MvccSnapshot[applied={T|T < 7333209182744010752}]
I20260925 11:04:33.159042 8799 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:04:33.159358 8799 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:04:33.163086 8799 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Sending DeleteTablet for 3 replicas of tablet 748a635605b242259cd5363e49e4c16f
I20260925 11:04:33.164250 8921 tablet_service.cc:1514] Processing DeleteTablet for tablet 748a635605b242259cd5363e49e4c16f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:04:33 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:33738
I20260925 11:04:33.164320 9054 tablet_service.cc:1514] Processing DeleteTablet for tablet 748a635605b242259cd5363e49e4c16f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:04:33 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:43800
I20260925 11:04:33.164656 9186 tablet_service.cc:1514] Processing DeleteTablet for tablet 748a635605b242259cd5363e49e4c16f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:04:33 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:35686
I20260925 11:04:33.165771 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 8838
I20260925 11:04:33.176651 9291 tablet_replica.cc:330] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: stopping tablet replica
I20260925 11:04:33.177157 9291 raft_consensus.cc:2229] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:04:33.177767 9291 raft_consensus.cc:2258] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:04:33.178750 9289 tablet_replica.cc:330] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: stopping tablet replica
I20260925 11:04:33.179186 9289 raft_consensus.cc:2229] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:04:33.180022 9289 raft_consensus.cc:2258] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:04:33.181313 9290 tablet_replica.cc:330] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: stopping tablet replica
I20260925 11:04:33.181740 9290 raft_consensus.cc:2229] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:04:33.182878 9289 ts_tablet_manager.cc:1905] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:04:33.183485 9291 ts_tablet_manager.cc:1905] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:04:33.187144 9290 raft_consensus.cc:2258] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:04:33.187494 9262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.166912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33748 (local address 127.24.159.193:45269)
0925 11:04:33.167095 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:33.167102 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:33.167134 (+ 32us) server_negotiation.cc:407] Connection header received
0925 11:04:33.167228 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:33.167234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:33.167316 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:33.167421 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:33.169406 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.170719 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.172844 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.173597 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.173720 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:33.178424 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:33.178443 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:33.178462 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:33.178507 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:33.182946 (+ 4439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:33.183858 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:33.183865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:33.183868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:33.184018 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:33.184526 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:33.184531 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:33.184534 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:33.184999 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:33.186408 (+ 1409us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:33.186552 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 11:04:33.186771 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:04:33.188090 8982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:33.190335 9291 ts_tablet_manager.cc:1918] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:04:33.190505 9291 log.cc:1196] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/748a635605b242259cd5363e49e4c16f
I20260925 11:04:33.190922 9289 ts_tablet_manager.cc:1918] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:04:33.191165 9289 log.cc:1196] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/748a635605b242259cd5363e49e4c16f
I20260925 11:04:33.191459 9291 ts_tablet_manager.cc:1939] T 748a635605b242259cd5363e49e4c16f P 7f19df8f16ff4e2d94ed1a6fbd5b718b: Deleting consensus metadata
I20260925 11:04:33.191846 9289 ts_tablet_manager.cc:1939] T 748a635605b242259cd5363e49e4c16f P 7131beabbe6f4b559cc1f3ef7c98fcd9: Deleting consensus metadata
I20260925 11:04:33.193490 8787 catalog_manager.cc:4841] TS 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031): tablet 748a635605b242259cd5363e49e4c16f (table test-table [id=f8f9c12e2af3424a8d4659051fc42a27]) successfully deleted
I20260925 11:04:33.193603 8786 catalog_manager.cc:4841] TS 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729): tablet 748a635605b242259cd5363e49e4c16f (table test-table [id=f8f9c12e2af3424a8d4659051fc42a27]) successfully deleted
I20260925 11:04:33.288113 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 8838
W20260925 11:04:33.309106 8786 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.159.193:45269 (error 108)
I20260925 11:04:33.309329 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 8991
I20260925 11:04:33.325127 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.324706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.0.0.1:33756)
0925 11:04:33.324885 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:33.324987 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:45269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 11:04:33.325419 8786 catalog_manager.cc:4584] TS 0177b285cbf44f1d9a77f65e15555b59 (127.24.159.193:45269): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 748a635605b242259cd5363e49e4c16f: Network error: Client connection negotiation failed: client connection to 127.24.159.193:45269: connect: Connection refused (error 111)
I20260925 11:04:33.327491 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.310283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.24.159.194:42729)
0925 11:04:33.310482 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:33.310489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:33.310520 (+ 31us) server_negotiation.cc:407] Connection header received
0925 11:04:33.310650 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:33.310656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:33.310751 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:33.310869 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:33.312615 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.313911 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.315949 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.316693 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.316824 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:33.320213 (+ 3389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:33.320232 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:33.320253 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:33.320294 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:33.324355 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:33.325250 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:33.325259 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:33.325263 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:33.325445 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:33.325850 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:33.325855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:33.325859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:33.326299 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:33.326584 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:33.326907 (+ 323us) server_negotiation.cc:299] Negotiation successful
0925 11:04:33.327158 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 11:04:33.328292 9114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:33.379240 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.378712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.0.0.1:33770)
0925 11:04:33.378940 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:33.379076 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:45269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260925 11:04:33.380298 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.379913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.0.0.1:33784)
0925 11:04:33.380094 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:33.380185 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:45269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260925 11:04:33.441313 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.440821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.0.0.1:33794)
0925 11:04:33.441008 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:33.441098 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:45269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 11:04:33.441978 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 8991
I20260925 11:04:33.444535 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.441969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.0.0.1:33800)
0925 11:04:33.444337 (+ 2368us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:33.444413 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:45269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2264}
I20260925 11:04:33.461390 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 9123
I20260925 11:04:33.475133 9265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.462163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35698 (local address 127.24.159.195:46031)
0925 11:04:33.462328 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:33.462332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:33.462389 (+ 57us) server_negotiation.cc:407] Connection header received
0925 11:04:33.462534 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:33.462540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:33.462621 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:33.462731 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:33.463961 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.465012 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.466431 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:33.466824 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:33.466919 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:33.470124 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:33.470139 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:33.470155 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:33.470227 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:33.472819 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:33.473416 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:33.473424 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:33.473426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:33.473578 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:33.473859 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:33.473862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:33.473864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:33.474184 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:33.474441 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:33.474661 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 11:04:33.474823 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:04:33.475729 9246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:33.524920 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.524429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.0.0.1:33804)
0925 11:04:33.524657 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:33.524752 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:45269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260925 11:04:33.525942 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:33.525555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:45269 (local address 127.0.0.1:33812)
0925 11:04:33.525762 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:33.525838 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:45269: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260925 11:04:33.563679 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 9123
I20260925 11:04:33.577611 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 8767
I20260925 11:04:33.578696 8830 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:04:33.690171 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 8767
I20260925 11:04:53.707871 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36321
--webserver_interface=127.24.159.254
--webserver_port=35233
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:36321
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:04:54.022468 9297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:54.022778 9297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:54.022913 9297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:54.030248 9297 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260925 11:04:54.030419 9297 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260925 11:04:54.030485 9297 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260925 11:04:54.030591 9297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:04:54.030668 9297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:54.030772 9297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:54.030838 9297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:04:54.030912 9297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:04:54.043100 9297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:36321
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36321
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=35233
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:54.044353 9297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:54.045636 9297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:54.057582 9303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:54.057504 9306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:54.057542 9304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:54.058424 9297 server_base.cc:1034] running on GCE node
I20260925 11:04:54.059136 9297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:54.060528 9297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:54.061738 9297 hybrid_clock.cc:648] HybridClock initialized: now 1790334294061657 us; error 49 us; skew 500 ppm
I20260925 11:04:54.062026 9297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:54 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:54.068468 9297 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:04:54.069948 9297 webserver.cc:460] Webserver started at http://127.24.159.254:35233/ using document root <none> and password file <none>
I20260925 11:04:54.070669 9297 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:54.070762 9297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:54.077237 9297 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260925 11:04:54.080222 9314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:54.081525 9297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:04:54.081683 9297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "4dfd4f139783497983159ef299f8862a"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "35a53a08153ac3344839371c7e244746"
server_key_iv: "a186468ad714ba9da0e4b7478db8b2ee"
server_key_version: "encryptionkey@0"
I20260925 11:04:54.082660 9297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:54.104221 9297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:54.109776 9297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:54.110131 9297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:54.128463 9297 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:36321
I20260925 11:04:54.128475 9365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:36321 every 8 connection(s)
I20260925 11:04:54.129871 9297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:04:54.136646 9366 sys_catalog.cc:263] Verifying existing consensus state
I20260925 11:04:54.139624 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 9297
I20260925 11:04:54.140076 9366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Bootstrap starting.
I20260925 11:04:54.140619 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:45269
--local_ip_for_outbound_sockets=127.24.159.193
--tserver_master_addrs=127.24.159.254:36321
--webserver_port=35171
--webserver_interface=127.24.159.193
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:04:54.162773 9366 log.cc:824] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:54.179421 9366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 11:04:54.179920 9366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Bootstrap complete.
I20260925 11:04:54.189666 9366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:54.190622 9366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dfd4f139783497983159ef299f8862a, State: Initialized, Role: FOLLOWER
I20260925 11:04:54.191187 9366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:54.191397 9366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:04:54.191501 9366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:04:54.191670 9366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 2 FOLLOWER]: Advancing to term 3
I20260925 11:04:54.195402 9366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:54.195858 9366 leader_election.cc:302] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dfd4f139783497983159ef299f8862a; no voters:
I20260925 11:04:54.196460 9366 leader_election.cc:288] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260925 11:04:54.196594 9370 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 3 FOLLOWER]: Leader election won for term 3
I20260925 11:04:54.201824 9370 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [term 3 LEADER]: Becoming Leader. State: Replica: 4dfd4f139783497983159ef299f8862a, State: Running, Role: LEADER
I20260925 11:04:54.202556 9370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } }
I20260925 11:04:54.203999 9366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:04:54.204962 9372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "4dfd4f139783497983159ef299f8862a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } } }
I20260925 11:04:54.205308 9372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: This master's current role is: LEADER
I20260925 11:04:54.209884 9371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4dfd4f139783497983159ef299f8862a. Latest consensus state: current_term: 3 leader_uuid: "4dfd4f139783497983159ef299f8862a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dfd4f139783497983159ef299f8862a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36321 } } }
I20260925 11:04:54.210228 9371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a [sys.catalog]: This master's current role is: LEADER
I20260925 11:04:54.211766 9375 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:04:54.220436 9375 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:04:54.221827 9375 catalog_manager.cc:1259] Loaded cluster ID: 1afaa20f8d9d4624ac2623a25a70813d
I20260925 11:04:54.221952 9375 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:04:54.225858 9375 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:04:54.227133 9387 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 11:04:54.227479 9375 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Loaded TSK: 0
I20260925 11:04:54.227897 9375 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4dfd4f139783497983159ef299f8862a: Loaded TSK: 1
F20260925 11:04:54.228977 9375 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9cf9de59ad google::LogMessage::Fail() at ??:0
@ 0x7f9cf9de888a google::LogMessage::SendToLog() at ??:0
@ 0x7f9cf9de54de google::LogMessage::Flush() at ??:0
@ 0x7f9cf9de6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9cfdb497d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9cfdb3873e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f9d0f8181c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9d0f817783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9d0f8a32a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f9d0f8a30c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f9d0f8ed5e9 std::function<>::operator()() at ??:0
@ 0x7f9d0f81bcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f9d0f819c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9d0f8a1610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f9d0f8a1439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9cfca31345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9cfca3491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f9cfca34799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f9cfca0feaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9cffce96db start_thread at ??:0
@ 0x7f9cf8c7f61f clone at ??:0
W20260925 11:04:54.519701 9368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:54.519984 9368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:54.520098 9368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:54.527371 9368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:54.527491 9368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:54.527642 9368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:04:54.537475 9368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:45269
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=35171
--tserver_master_addrs=127.24.159.254:36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:54.538767 9368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:54.540063 9368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:54.552620 9413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:54.552594 9414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:54.552670 9416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:54.553723 9368 server_base.cc:1034] running on GCE node
I20260925 11:04:54.554353 9368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:54.555266 9368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:54.556442 9368 hybrid_clock.cc:648] HybridClock initialized: now 1790334294556385 us; error 52 us; skew 500 ppm
I20260925 11:04:54.556730 9368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:54 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:54.559727 9368 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:04:54.561115 9368 webserver.cc:460] Webserver started at http://127.24.159.193:35171/ using document root <none> and password file <none>
I20260925 11:04:54.561779 9368 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:54.561898 9368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:54.567008 9368 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:04:54.569746 9423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:54.571012 9368 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:04:54.571157 9368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "0177b285cbf44f1d9a77f65e15555b59"
format_stamp: "Formatted at 2026-09-25 11:04:08 on dist-test-slave-trms"
server_key: "efdae6a3d56cb43158273eadd173aff9"
server_key_iv: "dce3df2274c95aea157abcd8de535511"
server_key_version: "encryptionkey@0"
I20260925 11:04:54.572058 9368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:54.582850 9368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:54.587003 9368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:54.587253 9368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:54.588214 9368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:04:54.590003 9430 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260925 11:04:54.595800 9368 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260925 11:04:54.595901 9368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260925 11:04:54.596022 9368 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260925 11:04:54.599109 9368 ts_tablet_manager.cc:610] Registered 1 tablets
I20260925 11:04:54.599186 9368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260925 11:04:54.599563 9430 tablet_bootstrap.cc:492] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Bootstrap starting.
I20260925 11:04:54.608110 9430 log.cc:824] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Log is configured to *not* fsync() on all Append() calls
I20260925 11:04:54.624667 9430 tablet_bootstrap.cc:492] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260925 11:04:54.625595 9430 tablet_bootstrap.cc:492] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Bootstrap complete.
I20260925 11:04:54.626246 9430 ts_tablet_manager.cc:1397] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Time spent bootstrapping tablet: real 0.027s user 0.025s sys 0.000s
I20260925 11:04:54.631899 9368 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:45269
I20260925 11:04:54.631935 9538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:45269 every 8 connection(s)
I20260925 11:04:54.633507 9368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:04:54.636737 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 9368
I20260925 11:04:54.636288 9430 raft_consensus.cc:348] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:54.637414 9430 raft_consensus.cc:729] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0177b285cbf44f1d9a77f65e15555b59, State: Initialized, Role: FOLLOWER
I20260925 11:04:54.637511 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:42729
--local_ip_for_outbound_sockets=127.24.159.194
--tserver_master_addrs=127.24.159.254:36321
--webserver_port=40633
--webserver_interface=127.24.159.194
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:04:54.638096 9430 consensus_queue.cc:260] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:54.639670 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:54.636681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:35045)
0925 11:04:54.638062 (+ 1381us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:54.638387 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":730,"thread_start_us":372,"threads_started":1}
I20260925 11:04:54.639818 9430 ts_tablet_manager.cc:1428] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59: Time spent starting tablet: real 0.013s user 0.006s sys 0.006s
W20260925 11:04:54.640265 9539 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:36321 (0 consecutive failures): Network error: Failed to ping master at 127.24.159.254:36321: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
I20260925 11:04:54.641114 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:54.640769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:58543)
0925 11:04:54.640913 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:54.640993 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:04:54.641896 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:54.641555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:39331)
0925 11:04:54.641677 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:54.641761 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 11:04:54.642108 9539 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260925 11:04:54.953328 9543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:54.953593 9543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:54.953694 9543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:54.961158 9543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:54.961263 9543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:54.961381 9543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:04:54.970968 9543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:42729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=40633
--tserver_master_addrs=127.24.159.254:36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:54.972151 9543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:54.973362 9543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:54.985714 9548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:54.985687 9549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:54.985879 9551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:54.986645 9543 server_base.cc:1034] running on GCE node
I20260925 11:04:54.987213 9543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:54.988157 9543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:54.989368 9543 hybrid_clock.cc:648] HybridClock initialized: now 1790334294989319 us; error 43 us; skew 500 ppm
I20260925 11:04:54.989672 9543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:54 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:54.992530 9543 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:04:54.993852 9543 webserver.cc:460] Webserver started at http://127.24.159.194:40633/ using document root <none> and password file <none>
I20260925 11:04:54.994536 9543 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:54.994649 9543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:54.999819 9543 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260925 11:04:55.002481 9558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:55.003788 9543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:04:55.003937 9543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "333f9069efa62ab79212374ef95afe9b"
server_key_iv: "cc64cd70ed719a2870faecb075e3bb84"
server_key_version: "encryptionkey@0"
I20260925 11:04:55.004875 9543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:55.032994 9543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:55.037308 9543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:55.037596 9543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:55.038573 9543 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:04:55.039932 9543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:04:55.040026 9543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:55.040141 9543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:04:55.040211 9543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:55.070432 9543 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:42729
I20260925 11:04:55.070497 9670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:42729 every 8 connection(s)
I20260925 11:04:55.071837 9543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:04:55.076922 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:55.074968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:40871)
0925 11:04:55.075742 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:55.075991 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":265,"threads_started":1}
W20260925 11:04:55.077692 9671 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:36321 (0 consecutive failures): Network error: Failed to ping master at 127.24.159.254:36321: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
I20260925 11:04:55.078619 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:55.078305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:40277)
0925 11:04:55.078438 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:55.078508 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260925 11:04:55.079463 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:55.079143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:49687)
0925 11:04:55.079280 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:55.079355 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 11:04:55.079703 9671 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 11:04:55.079802 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 9543
I20260925 11:04:55.080463 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:46031
--local_ip_for_outbound_sockets=127.24.159.195
--tserver_master_addrs=127.24.159.254:36321
--webserver_port=36873
--webserver_interface=127.24.159.195
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:38573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:04:55.391016 9674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:04:55.391296 9674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:04:55.391423 9674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:04:55.398671 9674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:04:55.398779 9674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:04:55.398932 9674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:04:55.408569 9674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:38573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:46031
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=36873
--tserver_master_addrs=127.24.159.254:36321
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:04:55.409900 9674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:04:55.411270 9674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:04:55.424042 9680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:55.424088 9679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:04:55.424481 9682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:04:55.424955 9674 server_base.cc:1034] running on GCE node
I20260925 11:04:55.425581 9674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:04:55.426580 9674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:04:55.427824 9674 hybrid_clock.cc:648] HybridClock initialized: now 1790334295427762 us; error 52 us; skew 500 ppm
I20260925 11:04:55.428144 9674 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:04:55 dist-test-slave-trms krb5kdc[8210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334295, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:04:55.431211 9674 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:04:55.432654 9674 webserver.cc:460] Webserver started at http://127.24.159.195:36873/ using document root <none> and password file <none>
I20260925 11:04:55.433323 9674 fs_manager.cc:359] Metadata directory not provided
I20260925 11:04:55.433462 9674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:04:55.438843 9674 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:04:55.441543 9689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:55.442971 9674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:04:55.443140 9674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b"
format_stamp: "Formatted at 2026-09-25 11:04:09 on dist-test-slave-trms"
server_key: "f6be516d2cc06adcc748b52cab6b2abe"
server_key_iv: "9a7c1658caded301a76907b9478cc736"
server_key_version: "encryptionkey@0"
I20260925 11:04:55.444097 9674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:04:55.480185 9674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:04:55.484302 9674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:04:55.484530 9674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:04:55.485455 9674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:04:55.486812 9674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:04:55.486904 9674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:55.486986 9674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:04:55.487038 9674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:04:55.516909 9674 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:46031
I20260925 11:04:55.516961 9801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:46031 every 8 connection(s)
I20260925 11:04:55.518441 9674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:04:55.521457 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 9674
I20260925 11:04:55.523346 9804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:55.521305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:56515)
0925 11:04:55.522097 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:55.522418 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":297,"threads_started":1}
W20260925 11:04:55.523972 9802 heartbeater.cc:644] Failed to heartbeat to 127.24.159.254:36321 (0 consecutive failures): Network error: Failed to ping master at 127.24.159.254:36321: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
I20260925 11:04:55.524745 9804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:55.524427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:33389)
0925 11:04:55.524570 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:55.524639 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:04:55.525451 9804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:55.525187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:38491)
0925 11:04:55.525287 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:55.525354 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260925 11:04:55.525660 9802 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260925 11:04:55.643296 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:55.642604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:42203)
0925 11:04:55.643029 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:55.643108 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":185,"threads_started":1}
I20260925 11:04:56.080960 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:56.080177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:54995)
0925 11:04:56.080646 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:56.080734 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":242,"threads_started":1}
I20260925 11:04:56.246896 9808 raft_consensus.cc:482] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:04:56.247205 9808 raft_consensus.cc:504] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:56.248672 9808 leader_election.cc:288] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729), 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031)
Sep 25 11:04:56 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
Sep 25 11:04:56 dist-test-slave-trms krb5kdc[8210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334294, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:04:56.271896 9812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:56.249259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:52371 (local address 127.24.159.195:46031)
0925 11:04:56.251382 (+ 2123us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:56.251397 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:56.251617 (+ 220us) server_negotiation.cc:407] Connection header received
0925 11:04:56.253041 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:56.253050 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:56.253390 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:56.253661 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:56.255713 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:56.256967 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:56.259594 (+ 2627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:56.260574 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:56.260756 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:56.265096 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:56.265107 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:56.265127 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:56.265174 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:56.268166 (+ 2992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:56.269064 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:56.269071 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:56.269076 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:56.269234 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:56.269666 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:56.269669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:56.269672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:56.270242 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:56.270418 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:56.271153 (+ 735us) server_negotiation.cc:299] Negotiation successful
0925 11:04:56.271479 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1955,"thread_start_us":1399,"threads_started":1}
I20260925 11:04:56.271910 9811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:56.249151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:33749 (local address 127.24.159.194:42729)
0925 11:04:56.251323 (+ 2172us) server_negotiation.cc:206] Beginning negotiation
0925 11:04:56.251340 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0925 11:04:56.251391 (+ 51us) server_negotiation.cc:407] Connection header received
0925 11:04:56.253003 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:04:56.253012 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:04:56.253392 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:04:56.253692 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:04:56.255717 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:56.257001 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:56.259590 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:04:56.260574 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:04:56.260756 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:56.265292 (+ 4536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:04:56.265299 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:04:56.265312 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:04:56.265353 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:04:56.268311 (+ 2958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:56.269069 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:56.269075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:56.269080 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:56.269231 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:04:56.269648 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:04:56.269654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:04:56.269657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:04:56.270179 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:04:56.270376 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0925 11:04:56.271158 (+ 782us) server_negotiation.cc:299] Negotiation successful
0925 11:04:56.271535 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1972,"thread_start_us":1803,"threads_started":1}
I20260925 11:04:56.271925 9810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:56.248942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:46031 (local address 127.24.159.193:52371)
0925 11:04:56.251512 (+ 2570us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:56.251529 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:56.251653 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:56.254106 (+ 2453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:56.254109 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:56.254788 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:04:56.255531 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:56.255550 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:56.257203 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:56.257207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:56.259458 (+ 2251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:56.259472 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:56.260786 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:56.260794 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:56.261156 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:56.261946 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:04:56.261984 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:04:56.264777 (+ 2793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:04:56.268329 (+ 3552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:56.268338 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:56.268353 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:56.268916 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:56.269346 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:56.269352 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:56.269355 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:56.269549 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:56.270361 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:04:56.270366 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:04:56.270671 (+ 305us) client_negotiation.cc:769] Sending connection context
0925 11:04:56.270909 (+ 238us) client_negotiation.cc:240] Negotiation successful
0925 11:04:56.271437 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2465,"mutex_wait_us":320,"spinlock_wait_cycles":36480,"thread_start_us":2326,"threads_started":1}
I20260925 11:04:56.271951 9809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:56.248877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:42729 (local address 127.24.159.193:33749)
0925 11:04:56.249519 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:56.249565 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0925 11:04:56.250170 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:04:56.253846 (+ 3676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:04:56.253857 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:04:56.254789 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:04:56.255531 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:56.255550 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:56.257197 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:56.257203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:56.259457 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:04:56.259472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:04:56.260801 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:04:56.260806 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:04:56.261152 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:04:56.261952 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:04:56.261984 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:04:56.265053 (+ 3069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:04:56.268463 (+ 3410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:56.268466 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:56.268469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:56.268928 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:56.269337 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:04:56.269342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:04:56.269346 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:04:56.269539 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:04:56.270332 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:04:56.270340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:04:56.270672 (+ 332us) client_negotiation.cc:769] Sending connection context
0925 11:04:56.270908 (+ 236us) client_negotiation.cc:240] Negotiation successful
0925 11:04:56.271573 (+ 665us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":419,"spinlock_wait_cycles":41984,"thread_start_us":509,"threads_started":1}
W20260925 11:04:56.276022 9426 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
W20260925 11:04:56.276312 9427 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
I20260925 11:04:56.276430 9427 leader_election.cc:302] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59; no voters: 7131beabbe6f4b559cc1f3ef7c98fcd9, 7f19df8f16ff4e2d94ed1a6fbd5b718b
I20260925 11:04:56.276775 9808 raft_consensus.cc:2726] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 11:04:56.526964 9813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:56.526127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:42263)
0925 11:04:56.526574 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:56.526723 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":225,"threads_started":1}
I20260925 11:04:56.644527 9810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:56.644132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:43783)
0925 11:04:56.644298 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:56.644390 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260925 11:04:57.082625 9814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:57.081852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:60151)
0925 11:04:57.082375 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:57.082459 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":267,"threads_started":1}
I20260925 11:04:57.528484 9815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:57.527803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:48081)
0925 11:04:57.528252 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:57.528332 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":241,"threads_started":1}
I20260925 11:04:57.646085 9816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:57.645368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:33521)
0925 11:04:57.645823 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:57.645914 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":231,"threads_started":1}
I20260925 11:04:58.084139 9817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:58.083468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:48433)
0925 11:04:58.083888 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:58.083970 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":214,"threads_started":1}
I20260925 11:04:58.296674 9818 raft_consensus.cc:482] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:04:58.296859 9818 raft_consensus.cc:504] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:58.297542 9818 leader_election.cc:288] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729), 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031)
W20260925 11:04:58.298579 9427 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
W20260925 11:04:58.298758 9426 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
I20260925 11:04:58.298955 9426 leader_election.cc:302] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59; no voters: 7131beabbe6f4b559cc1f3ef7c98fcd9, 7f19df8f16ff4e2d94ed1a6fbd5b718b
I20260925 11:04:58.299233 9818 raft_consensus.cc:2726] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 11:04:58.530159 9819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:58.529431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:35697)
0925 11:04:58.529925 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:58.530004 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":287,"threads_started":1}
I20260925 11:04:58.647688 9820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:58.646864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:46327)
0925 11:04:58.647316 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:58.647422 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":210,"threads_started":1}
I20260925 11:04:59.085676 9821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:59.085004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:44183)
0925 11:04:59.085414 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:59.085503 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":209,"threads_started":1}
I20260925 11:04:59.531821 9822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:59.531096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:57759)
0925 11:04:59.531564 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:59.531645 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260925 11:04:59.649399 9823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:04:59.648628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:34211)
0925 11:04:59.649125 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0925 11:04:59.649208 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":239,"threads_started":1}
I20260925 11:04:59.960805 9824 raft_consensus.cc:482] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:04:59.960988 9824 raft_consensus.cc:504] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:04:59.961634 9824 leader_election.cc:288] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729), 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031)
W20260925 11:04:59.962534 9426 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
W20260925 11:04:59.962708 9427 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
I20260925 11:04:59.962853 9427 leader_election.cc:302] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59; no voters: 7131beabbe6f4b559cc1f3ef7c98fcd9, 7f19df8f16ff4e2d94ed1a6fbd5b718b
I20260925 11:04:59.963086 9824 raft_consensus.cc:2726] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 11:05:00.087170 9825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:00.086499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:36431)
0925 11:05:00.086931 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:00.087010 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":220,"threads_started":1}
I20260925 11:05:00.533411 9826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:00.532704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:60455)
0925 11:05:00.533150 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:00.533243 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":219,"threads_started":1}
I20260925 11:05:00.650928 9827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:00.650250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:45107)
0925 11:05:00.650674 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:00.650767 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":202,"threads_started":1}
I20260925 11:05:01.088897 9828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:01.088105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:54953)
0925 11:05:01.088609 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:01.088707 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":255,"threads_started":1}
I20260925 11:05:01.535254 9829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:01.534455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:51605)
0925 11:05:01.534982 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:01.535074 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":295,"threads_started":1}
I20260925 11:05:01.652446 9830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:01.651744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:51335)
0925 11:05:01.652170 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:01.652259 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":223,"threads_started":1}
I20260925 11:05:02.090723 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:02.089857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:40903)
0925 11:05:02.090427 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:02.090525 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":247,"threads_started":1}
I20260925 11:05:02.537089 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:02.536229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:39405)
0925 11:05:02.536793 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:02.536898 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":240,"threads_started":1}
I20260925 11:05:02.654078 9833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:02.653322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:50047)
0925 11:05:02.653797 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:02.653892 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":268,"threads_started":1}
I20260925 11:05:03.092453 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:03.091670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:46359)
0925 11:05:03.092149 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:03.092261 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":244,"threads_started":1}
I20260925 11:05:03.393324 9835 raft_consensus.cc:482] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:05:03.393507 9835 raft_consensus.cc:504] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:05:03.394150 9835 leader_election.cc:288] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729), 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031)
W20260925 11:05:03.395128 9426 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
W20260925 11:05:03.395316 9427 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
I20260925 11:05:03.395467 9427 leader_election.cc:302] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59; no voters: 7131beabbe6f4b559cc1f3ef7c98fcd9, 7f19df8f16ff4e2d94ed1a6fbd5b718b
I20260925 11:05:03.395709 9835 raft_consensus.cc:2726] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 11:05:03.538849 9836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:03.537978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:54117)
0925 11:05:03.538504 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:03.538597 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":272,"threads_started":1}
I20260925 11:05:03.655779 9837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:03.654969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:40515)
0925 11:05:03.655439 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:03.655550 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":229,"threads_started":1}
I20260925 11:05:04.094053 9838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:04.093330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:45357)
0925 11:05:04.093801 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:04.093895 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":231,"threads_started":1}
I20260925 11:05:04.540504 9839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:04.539769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:56237)
0925 11:05:04.540253 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:04.540351 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":235,"threads_started":1}
I20260925 11:05:04.657579 9840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:04.656822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:55357)
0925 11:05:04.657302 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:04.657415 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":243,"threads_started":1}
I20260925 11:05:05.095801 9841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:05.095039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:38139)
0925 11:05:05.095527 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:05.095616 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":262,"threads_started":1}
I20260925 11:05:05.542390 9842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:05.541506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:57381)
0925 11:05:05.542030 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:05.542133 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":265,"threads_started":1}
I20260925 11:05:05.659381 9843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:05.658653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:46075)
0925 11:05:05.659106 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:05.659198 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":231,"threads_started":1}
I20260925 11:05:06.097548 9844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:06.096799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:48367)
0925 11:05:06.097287 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:06.097388 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":233,"threads_started":1}
I20260925 11:05:06.306067 9845 raft_consensus.cc:482] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:05:06.306290 9845 raft_consensus.cc:504] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0177b285cbf44f1d9a77f65e15555b59" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 45269 } } peers { permanent_uuid: "7131beabbe6f4b559cc1f3ef7c98fcd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 42729 } } peers { permanent_uuid: "7f19df8f16ff4e2d94ed1a6fbd5b718b" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 46031 } }
I20260925 11:05:06.306910 9845 leader_election.cc:288] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729), 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031)
W20260925 11:05:06.307765 9426 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7131beabbe6f4b559cc1f3ef7c98fcd9 (127.24.159.194:42729): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
W20260925 11:05:06.307926 9427 leader_election.cc:341] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7f19df8f16ff4e2d94ed1a6fbd5b718b (127.24.159.195:46031): Not found: Tablet not found: 748a635605b242259cd5363e49e4c16f
I20260925 11:05:06.308075 9427 leader_election.cc:302] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0177b285cbf44f1d9a77f65e15555b59; no voters: 7131beabbe6f4b559cc1f3ef7c98fcd9, 7f19df8f16ff4e2d94ed1a6fbd5b718b
I20260925 11:05:06.308346 9845 raft_consensus.cc:2726] T 748a635605b242259cd5363e49e4c16f P 0177b285cbf44f1d9a77f65e15555b59 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260925 11:05:06.544096 9846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:06.543364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:32917)
0925 11:05:06.543827 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:06.543931 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":228,"threads_started":1}
I20260925 11:05:06.660990 9847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:06.660196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:54891)
0925 11:05:06.660664 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:06.660763 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":222,"threads_started":1}
I20260925 11:05:07.099403 9848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:07.098592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:55427)
0925 11:05:07.099087 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:07.099184 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":268,"threads_started":1}
I20260925 11:05:07.545704 9849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:07.544970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:43439)
0925 11:05:07.545436 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:07.545517 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":251,"threads_started":1}
I20260925 11:05:07.662490 9850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:07.661762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:59467)
0925 11:05:07.662227 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:07.662317 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":223,"threads_started":1}
I20260925 11:05:08.101122 9851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:08.100374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:55175)
0925 11:05:08.100850 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:08.100948 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":243,"threads_started":1}
I20260925 11:05:08.547369 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:08.546628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:46161)
0925 11:05:08.547086 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:08.547192 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":231,"threads_started":1}
I20260925 11:05:08.663995 9853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:08.663269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:32905)
0925 11:05:08.663730 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:08.663820 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":213,"threads_started":1}
I20260925 11:05:09.102658 9854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:09.101920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:52639)
0925 11:05:09.102381 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:09.102467 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":261,"threads_started":1}
I20260925 11:05:09.548856 9855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:09.548145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:47007)
0925 11:05:09.548556 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:09.548659 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":218,"threads_started":1}
I20260925 11:05:09.665927 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:09.665118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.193:44833)
0925 11:05:09.665640 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:09.665744 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":261,"threads_started":1}
I20260925 11:05:10.104449 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:10.103672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.194:38299)
0925 11:05:10.104162 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:10.104270 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":232,"threads_started":1}
I20260925 11:05:10.528231 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 9368
I20260925 11:05:10.541936 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:10.529337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59278 (local address 127.24.159.193:45269)
0925 11:05:10.529833 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:10.529841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:10.529866 (+ 25us) server_negotiation.cc:407] Connection header received
0925 11:05:10.529927 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:10.529933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:10.530145 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:10.530320 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:10.531637 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:10.532638 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:10.533848 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:10.534454 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:10.534544 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:10.536973 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:10.536984 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:10.536997 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:10.537029 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:10.539643 (+ 2614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:10.540267 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:10.540273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:10.540275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:10.540397 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:10.540706 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:10.540709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:10.540711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:10.541041 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:10.541259 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:10.541484 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 11:05:10.541683 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":239,"threads_started":1}
I20260925 11:05:10.543303 9534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:10.550489 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:10.549772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36321 (local address 127.24.159.195:57991)
0925 11:05:10.550240 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:10.550337 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.254:36321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":225,"threads_started":1}
I20260925 11:05:10.631752 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 9368
I20260925 11:05:10.645723 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 9543
I20260925 11:05:10.658406 9861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:10.646448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.24.159.194:42729)
0925 11:05:10.646860 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:10.646865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:10.646881 (+ 16us) server_negotiation.cc:407] Connection header received
0925 11:05:10.646936 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:10.646939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:10.646999 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:10.647091 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:10.648482 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:10.649319 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:10.650567 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:10.650987 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:10.651169 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:10.653491 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:10.653496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:10.653499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:10.653532 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:10.656005 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:10.656686 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:10.656690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:10.656692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:10.656847 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:10.657144 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:10.657147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:10.657149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:10.657405 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:10.657626 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:10.657826 (+ 200us) server_negotiation.cc:299] Negotiation successful
0925 11:05:10.657990 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":212,"threads_started":1}
I20260925 11:05:10.659050 9666 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:10.741340 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 9543
I20260925 11:05:10.755085 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 9674
I20260925 11:05:10.767338 9863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:10.755942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47682 (local address 127.24.159.195:46031)
0925 11:05:10.756488 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:10.756492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:10.756509 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:05:10.756564 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:10.756568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:10.756627 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:10.756715 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:10.757982 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:10.758787 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:10.759930 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:10.760372 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:10.760462 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:10.762831 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:10.762837 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:10.762840 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:10.762872 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:10.765245 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:10.765807 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:10.765810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:10.765813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:10.765949 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:10.766248 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:10.766251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:10.766253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:10.766490 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:10.766691 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:10.766889 (+ 198us) server_negotiation.cc:299] Negotiation successful
0925 11:05:10.767050 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":316,"threads_started":1}
I20260925 11:05:10.768013 9797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:10.850399 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 9674
2026-09-25T11:05:10Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64159 ms)
[----------] 30 tests from SecurityITest (288308 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:10 dist-test-slave-trms krb5kdc[9867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:10 dist-test-slave-trms krb5kdc[9867](info): set up 2 sockets
Sep 25 11:05:10 dist-test-slave-trms krb5kdc[9867](info): commencing operation
krb5kdc: starting...
W20260925 11:05:11.862248 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:12 dist-test-slave-trms krb5kdc[9867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:05:12.093453 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:40219
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:40219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:12.412297 9880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:12.412598 9880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:12.412719 9880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:12.419889 9880 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:12.420022 9880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:12.420113 9880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:12.420197 9880 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:12.420255 9880 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:12.430068 9880 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:40219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:40219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:12.431407 9880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:12.432698 9880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:12.444038 9885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:12.443996 9886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:12.444083 9888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:12.444939 9880 server_base.cc:1034] running on GCE node
I20260925 11:05:12.445709 9880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:12.447105 9880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:12.448338 9880 hybrid_clock.cc:648] HybridClock initialized: now 1790334312448277 us; error 47 us; skew 500 ppm
I20260925 11:05:12.448683 9880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:12 dist-test-slave-trms krb5kdc[9867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334312, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:12.455089 9880 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:05:12.456633 9880 webserver.cc:460] Webserver started at http://127.24.159.254:39809/ using document root <none> and password file <none>
I20260925 11:05:12.457322 9880 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:12.457439 9880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:12.457731 9880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:12.460701 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "310a75de14ef498a92094f35672b3041"
format_stamp: "Formatted at 2026-09-25 11:05:12 on dist-test-slave-trms"
server_key: "6accd632b7925164cf986222711f9c7b"
server_key_iv: "63b82deaf01c5a629e433c8440d51e96"
server_key_version: "encryptionkey@0"
I20260925 11:05:12.461426 9880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "310a75de14ef498a92094f35672b3041"
format_stamp: "Formatted at 2026-09-25 11:05:12 on dist-test-slave-trms"
server_key: "6accd632b7925164cf986222711f9c7b"
server_key_iv: "63b82deaf01c5a629e433c8440d51e96"
server_key_version: "encryptionkey@0"
I20260925 11:05:12.466390 9880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 11:05:12.469835 9895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:12.471161 9880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:05:12.471319 9880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "310a75de14ef498a92094f35672b3041"
format_stamp: "Formatted at 2026-09-25 11:05:12 on dist-test-slave-trms"
server_key: "6accd632b7925164cf986222711f9c7b"
server_key_iv: "63b82deaf01c5a629e433c8440d51e96"
server_key_version: "encryptionkey@0"
I20260925 11:05:12.471499 9880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:12.492437 9880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:12.496321 9880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:12.496574 9880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:12.513706 9880 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:40219
I20260925 11:05:12.513726 9946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:40219 every 8 connection(s)
I20260925 11:05:12.515091 9880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:12.518832 9947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:12.523646 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 9880
I20260925 11:05:12.523886 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:12.524667 25215 external_mini_cluster.cc:1428] Setting key 40e6fc189db87b4ee5b248085b35b651
I20260925 11:05:12.530207 9947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041: Bootstrap starting.
I20260925 11:05:12.533831 9947 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:12.535110 9947 log.cc:824] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:12.538465 9947 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041: No bootstrap required, opened a new log
Sep 25 11:05:12 dist-test-slave-trms krb5kdc[9867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:12.547379 9947 raft_consensus.cc:348] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "310a75de14ef498a92094f35672b3041" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 40219 } }
I20260925 11:05:12.547762 9947 raft_consensus.cc:374] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:12.547863 9947 raft_consensus.cc:729] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 310a75de14ef498a92094f35672b3041, State: Initialized, Role: FOLLOWER
I20260925 11:05:12.548192 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:12.526778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.24.159.254:40219)
0925 11:05:12.527592 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:12.527606 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:12.527658 (+ 52us) server_negotiation.cc:407] Connection header received
0925 11:05:12.529394 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:12.529417 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:12.529815 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:12.530144 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:12.531980 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:12.533267 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:12.535110 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:12.536054 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:12.536274 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:12.540862 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:12.540870 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:12.540884 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:12.540921 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:12.544054 (+ 3133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:12.544871 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:12.544878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:12.544885 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:12.545044 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:12.545450 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:12.545453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:12.545455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:12.545973 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:12.546256 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:12.546651 (+ 395us) server_negotiation.cc:299] Negotiation successful
0925 11:05:12.547013 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":386,"threads_started":1}
I20260925 11:05:12.548446 9947 consensus_queue.cc:260] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "310a75de14ef498a92094f35672b3041" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 40219 } }
I20260925 11:05:12.548674 9947 raft_consensus.cc:388] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:12.548804 9947 raft_consensus.cc:482] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:12.548956 9947 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:12.551262 9947 raft_consensus.cc:504] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "310a75de14ef498a92094f35672b3041" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 40219 } }
I20260925 11:05:12.551708 9947 leader_election.cc:302] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 310a75de14ef498a92094f35672b3041; no voters:
I20260925 11:05:12.552245 9947 leader_election.cc:288] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:12.552397 9952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:12.553419 9952 raft_consensus.cc:686] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [term 1 LEADER]: Becoming Leader. State: Replica: 310a75de14ef498a92094f35672b3041, State: Running, Role: LEADER
I20260925 11:05:12.553902 9952 consensus_queue.cc:237] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "310a75de14ef498a92094f35672b3041" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 40219 } }
I20260925 11:05:12.554332 9947 sys_catalog.cc:564] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:12.556401 9954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 310a75de14ef498a92094f35672b3041. Latest consensus state: current_term: 1 leader_uuid: "310a75de14ef498a92094f35672b3041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "310a75de14ef498a92094f35672b3041" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 40219 } } }
I20260925 11:05:12.556767 9954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:12.557240 9953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "310a75de14ef498a92094f35672b3041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "310a75de14ef498a92094f35672b3041" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 40219 } } }
I20260925 11:05:12.557509 9953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:12.558103 9960 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:12.564229 9960 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:12.575217 9960 catalog_manager.cc:1347] Generated new cluster ID: cd4e18ae5eff4fb49afeed81fb378ea6
I20260925 11:05:12.575325 9960 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:12.590770 9960 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:12.592665 9960 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:12.603935 9960 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041: Generated new TSK 0
I20260925 11:05:12.605055 9960 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:05:12.670213 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:40219
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:12.997116 9973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:12.997400 9973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:12.997530 9973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:13.004611 9973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:13.004730 9973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:13.004878 9973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:05:13.014693 9973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:40219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:13.015992 9973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:13.017295 9973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:13.029971 9978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:13.029939 9979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:13.030169 9981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:13.030761 9973 server_base.cc:1034] running on GCE node
I20260925 11:05:13.031347 9973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:13.032279 9973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:13.033484 9973 hybrid_clock.cc:648] HybridClock initialized: now 1790334313033438 us; error 39 us; skew 500 ppm
I20260925 11:05:13.033794 9973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:13 dist-test-slave-trms krb5kdc[9867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:13.036867 9973 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:05:13.038259 9973 webserver.cc:460] Webserver started at http://127.24.159.193:37447/ using document root <none> and password file <none>
I20260925 11:05:13.038920 9973 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:13.039028 9973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:13.039341 9973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:13.042157 9973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "55f38c650a854cb48fa65ea78a399294"
format_stamp: "Formatted at 2026-09-25 11:05:13 on dist-test-slave-trms"
server_key: "c274e9fdf97efab1198dd290675a4a3f"
server_key_iv: "31de8bdd6525e9ef4958d33141270f47"
server_key_version: "encryptionkey@0"
I20260925 11:05:13.042914 9973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "55f38c650a854cb48fa65ea78a399294"
format_stamp: "Formatted at 2026-09-25 11:05:13 on dist-test-slave-trms"
server_key: "c274e9fdf97efab1198dd290675a4a3f"
server_key_iv: "31de8bdd6525e9ef4958d33141270f47"
server_key_version: "encryptionkey@0"
I20260925 11:05:13.047943 9973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:05:13.051324 9988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:13.052556 9973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260925 11:05:13.052706 9973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "55f38c650a854cb48fa65ea78a399294"
format_stamp: "Formatted at 2026-09-25 11:05:13 on dist-test-slave-trms"
server_key: "c274e9fdf97efab1198dd290675a4a3f"
server_key_iv: "31de8bdd6525e9ef4958d33141270f47"
server_key_version: "encryptionkey@0"
I20260925 11:05:13.052881 9973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:13.071655 9973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:13.075878 9973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:13.076126 9973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:13.077073 9973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:13.078408 9973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:13.078477 9973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:13.078593 9973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:13.078651 9973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:13.106112 9973 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:39775
I20260925 11:05:13.106132 10100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:39775 every 8 connection(s)
I20260925 11:05:13.107523 9973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:05:13.116603 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 9973
I20260925 11:05:13.116783 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:05:13.117482 25215 external_mini_cluster.cc:1428] Setting key e85ec3d7d354d09b33a7f8ba4d706015
Sep 25 11:05:13 dist-test-slave-trms krb5kdc[9867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:13.127379 10103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:13.109995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:57617 (local address 127.24.159.254:40219)
0925 11:05:13.110462 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:13.110467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:13.111452 (+ 985us) server_negotiation.cc:407] Connection header received
0925 11:05:13.113287 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:13.113291 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:13.113358 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:13.113442 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:13.114914 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:13.115622 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:13.117219 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:13.117827 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:13.117963 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:13.121634 (+ 3671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:13.121640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:13.121644 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:13.121681 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:13.124465 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:13.125135 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:13.125141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:13.125145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:13.125291 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:13.125675 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:13.125680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:13.125682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:13.125993 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:13.126123 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:13.126843 (+ 720us) server_negotiation.cc:299] Negotiation successful
0925 11:05:13.127049 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":244,"threads_started":1}
I20260925 11:05:13.128158 10104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:13.110613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:40219 (local address 127.24.159.193:57617)
0925 11:05:13.111290 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:13.111328 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:13.113011 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:13.113636 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:13.113644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:13.114147 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:13.114756 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:13.114775 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:13.115748 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:13.115752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:13.117081 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:13.117094 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:13.118037 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:13.118042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:13.118358 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:13.119003 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:05:13.119036 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:13.121411 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:13.124590 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:13.124597 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:13.124610 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:13.125015 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:13.125380 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:13.125383 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:13.125385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:13.125564 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:13.126093 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:13.126099 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:13.126462 (+ 363us) client_negotiation.cc:769] Sending connection context
0925 11:05:13.126694 (+ 232us) client_negotiation.cc:240] Negotiation successful
0925 11:05:13.127073 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":250,"threads_started":1}
I20260925 11:05:13.129774 10101 heartbeater.cc:347] Connected to a master server at 127.24.159.254:40219
I20260925 11:05:13.130077 10101 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:13.130820 10101 heartbeater.cc:511] Master 127.24.159.254:40219 requested a full tablet report, sending...
I20260925 11:05:13.132963 9912 ts_manager.cc:194] Registered new tserver with Master: 55f38c650a854cb48fa65ea78a399294 (127.24.159.193:39775)
I20260925 11:05:13.134464 9912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:57617
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:05:13.170135 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:40219
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:13.490680 10107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:13.490950 10107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:13.491055 10107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:13.498126 10107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:13.498266 10107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:13.498400 10107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:05:13.507830 10107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:40219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:13.508999 10107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:13.510306 10107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:13.523211 10113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:13.523547 10112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:13.523789 10115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:13.524248 10107 server_base.cc:1034] running on GCE node
I20260925 11:05:13.524859 10107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:13.525830 10107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:13.527024 10107 hybrid_clock.cc:648] HybridClock initialized: now 1790334313526955 us; error 61 us; skew 500 ppm
I20260925 11:05:13.527333 10107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:13 dist-test-slave-trms krb5kdc[9867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:13.530351 10107 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:05:13.531819 10107 webserver.cc:460] Webserver started at http://127.24.159.194:39215/ using document root <none> and password file <none>
I20260925 11:05:13.532464 10107 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:13.532581 10107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:13.532878 10107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:13.535720 10107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "48c54bea818e4df794095f62233e2984"
format_stamp: "Formatted at 2026-09-25 11:05:13 on dist-test-slave-trms"
server_key: "f387f0dafdba67be20dc90d16f8beb7e"
server_key_iv: "29c5b3fb0c8f882569a1411c9c07dcdc"
server_key_version: "encryptionkey@0"
I20260925 11:05:13.536439 10107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "48c54bea818e4df794095f62233e2984"
format_stamp: "Formatted at 2026-09-25 11:05:13 on dist-test-slave-trms"
server_key: "f387f0dafdba67be20dc90d16f8beb7e"
server_key_iv: "29c5b3fb0c8f882569a1411c9c07dcdc"
server_key_version: "encryptionkey@0"
I20260925 11:05:13.541571 10107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:05:13.545024 10122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:13.546377 10107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:05:13.546531 10107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "48c54bea818e4df794095f62233e2984"
format_stamp: "Formatted at 2026-09-25 11:05:13 on dist-test-slave-trms"
server_key: "f387f0dafdba67be20dc90d16f8beb7e"
server_key_iv: "29c5b3fb0c8f882569a1411c9c07dcdc"
server_key_version: "encryptionkey@0"
I20260925 11:05:13.546721 10107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:13.570745 10107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:13.575052 10107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:13.575342 10107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:13.576342 10107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:13.577626 10107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:13.577703 10107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:13.577832 10107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:13.577886 10107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:13.607750 10107 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:41791
I20260925 11:05:13.607800 10234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:41791 every 8 connection(s)
I20260925 11:05:13.609133 10107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:05:13.612123 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 10107
I20260925 11:05:13.612298 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:05:13.613022 25215 external_mini_cluster.cc:1428] Setting key d9addaf0d7904d940af6bafb45a1c154
Sep 25 11:05:13 dist-test-slave-trms krb5kdc[9867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334313, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:13.630805 10103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:13.611631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:53017 (local address 127.24.159.254:40219)
0925 11:05:13.611839 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:13.611846 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:13.613179 (+ 1333us) server_negotiation.cc:407] Connection header received
0925 11:05:13.615410 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:13.615414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:13.615481 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:13.615560 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:13.617556 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:13.618303 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:13.620305 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:13.620700 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:13.620787 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:13.624725 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:13.624732 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:13.624737 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:13.624779 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:13.627824 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:13.628521 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:13.628527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:13.628530 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:13.628688 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:13.629096 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:13.629100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:13.629109 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:13.629433 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:13.629575 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:13.630271 (+ 696us) server_negotiation.cc:299] Negotiation successful
0925 11:05:13.630494 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:05:13.631677 10237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:13.612128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:40219 (local address 127.24.159.194:53017)
0925 11:05:13.612989 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:13.613044 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:13.615122 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:13.615840 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:13.615852 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:13.616553 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:13.617382 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:13.617410 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:13.618458 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:13.618463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:13.620185 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:13.620200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:13.620917 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:13.620924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:13.621193 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:13.621879 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:05:13.621907 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:13.624489 (+ 2582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:13.627953 (+ 3464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:13.627959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:13.627972 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:13.628408 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:13.628778 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:13.628788 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:13.628791 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:13.628987 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:13.629561 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:13.629567 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:13.629892 (+ 325us) client_negotiation.cc:769] Sending connection context
0925 11:05:13.630122 (+ 230us) client_negotiation.cc:240] Negotiation successful
0925 11:05:13.630546 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":306,"threads_started":1}
I20260925 11:05:13.633121 10235 heartbeater.cc:347] Connected to a master server at 127.24.159.254:40219
I20260925 11:05:13.633404 10235 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:13.634091 10235 heartbeater.cc:511] Master 127.24.159.254:40219 requested a full tablet report, sending...
I20260925 11:05:13.635591 9912 ts_manager.cc:194] Registered new tserver with Master: 48c54bea818e4df794095f62233e2984 (127.24.159.194:41791)
I20260925 11:05:13.636408 9912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:53017
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:05:13.666527 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:40219
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:41401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:13.987380 10240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:13.987691 10240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:13.987829 10240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:13.995059 10240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:13.995195 10240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:13.995347 10240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:05:14.005162 10240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:41401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:40219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:14.006532 10240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:14.007833 10240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:14.020516 10245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:14.020556 10248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:14.020475 10246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:14.021215 10240 server_base.cc:1034] running on GCE node
I20260925 11:05:14.021821 10240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:14.022816 10240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:14.024000 10240 hybrid_clock.cc:648] HybridClock initialized: now 1790334314023943 us; error 50 us; skew 500 ppm
I20260925 11:05:14.024290 10240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:14 dist-test-slave-trms krb5kdc[9867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334314, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:14.027305 10240 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:05:14.030558 10240 webserver.cc:460] Webserver started at http://127.24.159.195:33663/ using document root <none> and password file <none>
I20260925 11:05:14.031491 10240 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:14.031656 10240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:14.032039 10240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:14.035782 10240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "d5b727e4c16641ceae0c47a86b906e03"
format_stamp: "Formatted at 2026-09-25 11:05:14 on dist-test-slave-trms"
server_key: "4d8dd4600342ebe281d3e759451faa70"
server_key_iv: "e157f62ede306c30d1da53f3fb74e1e9"
server_key_version: "encryptionkey@0"
I20260925 11:05:14.036544 10240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "d5b727e4c16641ceae0c47a86b906e03"
format_stamp: "Formatted at 2026-09-25 11:05:14 on dist-test-slave-trms"
server_key: "4d8dd4600342ebe281d3e759451faa70"
server_key_iv: "e157f62ede306c30d1da53f3fb74e1e9"
server_key_version: "encryptionkey@0"
I20260925 11:05:14.041754 10240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:05:14.045287 10255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:14.046718 10240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:05:14.046891 10240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "d5b727e4c16641ceae0c47a86b906e03"
format_stamp: "Formatted at 2026-09-25 11:05:14 on dist-test-slave-trms"
server_key: "4d8dd4600342ebe281d3e759451faa70"
server_key_iv: "e157f62ede306c30d1da53f3fb74e1e9"
server_key_version: "encryptionkey@0"
I20260925 11:05:14.047071 10240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:14.074059 10240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:14.078630 10240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:14.078922 10240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:14.079945 10240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:14.081343 10240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:14.081418 10240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:14.081529 10240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:14.081580 10240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:14.112015 10240 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:45401
I20260925 11:05:14.112072 10367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:45401 every 8 connection(s)
I20260925 11:05:14.113294 10240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:05:14.118873 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 10240
I20260925 11:05:14.119035 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:05:14.119588 25215 external_mini_cluster.cc:1428] Setting key 67a7fe4a2968c1c8abf9cd736f35805a
Sep 25 11:05:14 dist-test-slave-trms krb5kdc[9867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334314, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:14.132887 10103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.115962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:48931 (local address 127.24.159.254:40219)
0925 11:05:14.116187 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.116194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.117383 (+ 1189us) server_negotiation.cc:407] Connection header received
0925 11:05:14.119330 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.119336 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.119440 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.119554 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:14.121217 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.122103 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.123724 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.124121 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.124218 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.127672 (+ 3454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:14.127677 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:14.127681 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:14.127712 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:14.129965 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.130692 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.130696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.130699 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.130821 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.131229 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.131232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.131234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.131491 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:14.131628 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.132323 (+ 695us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.132539 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 11:05:14.133994 10370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.116481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:40219 (local address 127.24.159.195:48931)
0925 11:05:14.117201 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:14.117241 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:14.119020 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:14.119770 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:14.119778 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:14.120349 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:14.121050 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.121070 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.122290 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.122296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.123596 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.123612 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.124299 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.124304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.124498 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.125067 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:05:14.125089 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:14.127467 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:14.130100 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:14.130109 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:14.130126 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:14.130574 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:14.130922 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:14.130927 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:14.130931 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:14.131119 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:14.131610 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:14.131617 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:14.131943 (+ 326us) client_negotiation.cc:769] Sending connection context
0925 11:05:14.132189 (+ 246us) client_negotiation.cc:240] Negotiation successful
0925 11:05:14.132696 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":274,"threads_started":1}
I20260925 11:05:14.135327 10368 heartbeater.cc:347] Connected to a master server at 127.24.159.254:40219
I20260925 11:05:14.135627 10368 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:14.136329 10368 heartbeater.cc:511] Master 127.24.159.254:40219 requested a full tablet report, sending...
I20260925 11:05:14.137468 10101 heartbeater.cc:503] Master 127.24.159.254:40219 was elected leader, sending a full tablet report...
I20260925 11:05:14.137826 9912 ts_manager.cc:194] Registered new tserver with Master: d5b727e4c16641ceae0c47a86b906e03 (127.24.159.195:45401)
I20260925 11:05:14.138748 9912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:48931
I20260925 11:05:14.147337 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:05:14.163295 10103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.151732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36072 (local address 127.24.159.254:40219)
0925 11:05:14.151919 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.151923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.152378 (+ 455us) server_negotiation.cc:407] Connection header received
0925 11:05:14.152517 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.152521 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.152584 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.152666 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:14.153950 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.154755 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.156120 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.156581 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.156674 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.158922 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:14.158927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:14.158931 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:14.158963 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:14.161156 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.161699 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.161703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.161705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.161820 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.162095 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.162098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.162100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.162452 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:14.162629 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.162887 (+ 258us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.163042 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260925 11:05:14.167691 9912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36072:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:05:14.174577 9912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:05:14.196262 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.186827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52706 (local address 127.24.159.195:45401)
0925 11:05:14.188070 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.188081 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.188383 (+ 302us) server_negotiation.cc:407] Connection header received
0925 11:05:14.188493 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.188501 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.188789 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.188957 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:14.189548 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.191445 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.193942 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.195158 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.195341 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.195526 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.195744 (+ 218us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.195915 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":821,"thread_start_us":676,"threads_started":1}
I20260925 11:05:14.196367 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.186571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:45401 (local address 127.0.0.1:52706)
0925 11:05:14.188114 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:14.188136 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:14.188254 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:14.188984 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:14.188991 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:14.189019 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:14.189369 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.189381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.191571 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.191577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.193761 (+ 2184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.193776 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.195341 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.195347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.195527 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.195577 (+ 50us) client_negotiation.cc:769] Sending connection context
0925 11:05:14.195691 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 11:05:14.195891 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1406,"mutex_wait_us":41,"thread_start_us":1415,"threads_started":1}
I20260925 11:05:14.196818 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.188314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39775 (local address 127.0.0.1:33442)
0925 11:05:14.188788 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:14.188809 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:14.188917 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:14.190035 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:14.190039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:14.190064 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:14.190473 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.190485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.192292 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.192297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.194144 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.194157 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.195979 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.195986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.196165 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.196201 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 11:05:14.196264 (+ 63us) client_negotiation.cc:240] Negotiation successful
0925 11:05:14.196405 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":107,"thread_start_us":275,"threads_started":1}
I20260925 11:05:14.197311 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.186180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:41791 (local address 127.0.0.1:54936)
0925 11:05:14.186946 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:14.187003 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:14.187206 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:14.189773 (+ 2567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:14.189778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:14.189801 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:14.190256 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.190269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.192207 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.192212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.194584 (+ 2372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.194599 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.196379 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.196385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.196574 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.196616 (+ 42us) client_negotiation.cc:769] Sending connection context
0925 11:05:14.196685 (+ 69us) client_negotiation.cc:240] Negotiation successful
0925 11:05:14.196812 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"mutex_wait_us":63,"thread_start_us":285,"threads_started":1}
I20260925 11:05:14.197685 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.186421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54936 (local address 127.24.159.194:41791)
0925 11:05:14.188499 (+ 2078us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.188508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.188536 (+ 28us) server_negotiation.cc:407] Connection header received
0925 11:05:14.189369 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.189378 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.189613 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.189769 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:14.190498 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.192055 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.194748 (+ 2693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.196135 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.196932 (+ 797us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.197108 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.197196 (+ 88us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.197364 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1931,"thread_start_us":1802,"threads_started":1}
I20260925 11:05:14.198060 10385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.188893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33442 (local address 127.24.159.193:39775)
0925 11:05:14.189500 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.189508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.189531 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:05:14.189617 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.189625 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.189860 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.190042 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:14.190656 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.192123 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.194338 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.195743 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.197287 (+ 1544us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.197525 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.197597 (+ 72us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.197760 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":315,"threads_started":1}
I20260925 11:05:14.200491 10303 tablet_service.cc:1467] Processing CreateTablet for tablet 9a62b542153044f18e188efdc3c8105e (DEFAULT_TABLE table=test-table [id=8c3a5b9963c94c4880af5f75e86ea47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:14.200949 10170 tablet_service.cc:1467] Processing CreateTablet for tablet 9a62b542153044f18e188efdc3c8105e (DEFAULT_TABLE table=test-table [id=8c3a5b9963c94c4880af5f75e86ea47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:14.201987 10303 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a62b542153044f18e188efdc3c8105e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:14.202106 10170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a62b542153044f18e188efdc3c8105e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:14.203801 10036 tablet_service.cc:1467] Processing CreateTablet for tablet 9a62b542153044f18e188efdc3c8105e (DEFAULT_TABLE table=test-table [id=8c3a5b9963c94c4880af5f75e86ea47a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:14.205183 10036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a62b542153044f18e188efdc3c8105e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:14.213344 10386 tablet_bootstrap.cc:492] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: Bootstrap starting.
I20260925 11:05:14.216620 10387 tablet_bootstrap.cc:492] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: Bootstrap starting.
I20260925 11:05:14.217276 10386 tablet_bootstrap.cc:654] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:14.218744 10386 log.cc:824] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:14.220774 10387 tablet_bootstrap.cc:654] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:14.221989 10387 log.cc:824] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:14.222054 10386 tablet_bootstrap.cc:492] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: No bootstrap required, opened a new log
I20260925 11:05:14.220175 10389 tablet_bootstrap.cc:492] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: Bootstrap starting.
I20260925 11:05:14.222363 10386 ts_tablet_manager.cc:1397] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260925 11:05:14.224855 10389 tablet_bootstrap.cc:654] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:14.226075 10389 log.cc:824] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:14.227700 10387 tablet_bootstrap.cc:492] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: No bootstrap required, opened a new log
I20260925 11:05:14.227973 10387 ts_tablet_manager.cc:1397] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.000s
I20260925 11:05:14.229348 10389 tablet_bootstrap.cc:492] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: No bootstrap required, opened a new log
I20260925 11:05:14.229591 10389 ts_tablet_manager.cc:1397] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.003s
I20260925 11:05:14.231055 10386 raft_consensus.cc:348] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } }
I20260925 11:05:14.231412 10386 raft_consensus.cc:374] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:14.231503 10386 raft_consensus.cc:729] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48c54bea818e4df794095f62233e2984, State: Initialized, Role: FOLLOWER
I20260925 11:05:14.232055 10386 consensus_queue.cc:260] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } }
I20260925 11:05:14.233075 10235 heartbeater.cc:503] Master 127.24.159.254:40219 was elected leader, sending a full tablet report...
I20260925 11:05:14.233726 10386 ts_tablet_manager.cc:1428] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260925 11:05:14.238361 10387 raft_consensus.cc:348] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } }
I20260925 11:05:14.238823 10387 raft_consensus.cc:374] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:14.238925 10387 raft_consensus.cc:729] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5b727e4c16641ceae0c47a86b906e03, State: Initialized, Role: FOLLOWER
I20260925 11:05:14.239606 10387 consensus_queue.cc:260] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } }
I20260925 11:05:14.239717 10389 raft_consensus.cc:348] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } }
I20260925 11:05:14.240123 10389 raft_consensus.cc:374] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:14.240227 10389 raft_consensus.cc:729] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55f38c650a854cb48fa65ea78a399294, State: Initialized, Role: FOLLOWER
I20260925 11:05:14.240728 10368 heartbeater.cc:503] Master 127.24.159.254:40219 was elected leader, sending a full tablet report...
I20260925 11:05:14.240805 10389 consensus_queue.cc:260] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } }
I20260925 11:05:14.241492 10387 ts_tablet_manager.cc:1428] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260925 11:05:14.242440 10389 ts_tablet_manager.cc:1428] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: Time spent starting tablet: real 0.013s user 0.009s sys 0.006s
I20260925 11:05:14.248090 10394 raft_consensus.cc:482] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:05:14.248337 10394 raft_consensus.cc:504] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } }
I20260925 11:05:14.250687 10394 leader_election.cc:288] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48c54bea818e4df794095f62233e2984 (127.24.159.194:41791), d5b727e4c16641ceae0c47a86b906e03 (127.24.159.195:45401)
I20260925 11:05:14.257174 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.250466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:36981 (local address 127.24.159.194:41791)
0925 11:05:14.250699 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.250708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.251553 (+ 845us) server_negotiation.cc:407] Connection header received
0925 11:05:14.251839 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.251845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.251926 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.252013 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:14.252623 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.253382 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.255388 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.256242 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.256354 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.256407 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.256792 (+ 385us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.256923 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:05:14.257265 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.251002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:40523 (local address 127.24.159.195:45401)
0925 11:05:14.251203 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.251208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.251964 (+ 756us) server_negotiation.cc:407] Connection header received
0925 11:05:14.252122 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.252127 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.252198 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.252288 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:14.252857 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.253688 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.255505 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.256412 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.256580 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.256658 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.256925 (+ 267us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.257045 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 11:05:14.257551 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.250406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:41791 (local address 127.24.159.193:36981)
0925 11:05:14.251422 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:14.251444 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:14.251679 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:14.252037 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:14.252068 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:14.252104 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:14.252474 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.252485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.253518 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.253523 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.255225 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.255239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.256429 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.256436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.256612 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.256665 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 11:05:14.256751 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 11:05:14.256845 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":858,"mutex_wait_us":4,"thread_start_us":662,"threads_started":1}
I20260925 11:05:14.257591 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.250896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:45401 (local address 127.24.159.193:40523)
0925 11:05:14.251865 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:14.251881 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:14.252004 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:14.252298 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:14.252302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:14.252321 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:14.252671 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.252684 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.253824 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.253828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.255371 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:14.255386 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.256610 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.256614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:14.256766 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.256797 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 11:05:14.256883 (+ 86us) client_negotiation.cc:240] Negotiation successful
0925 11:05:14.256994 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":16,"thread_start_us":710,"threads_started":1}
I20260925 11:05:14.258422 10323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a62b542153044f18e188efdc3c8105e" candidate_uuid: "55f38c650a854cb48fa65ea78a399294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b727e4c16641ceae0c47a86b906e03" is_pre_election: true
I20260925 11:05:14.258620 10190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a62b542153044f18e188efdc3c8105e" candidate_uuid: "55f38c650a854cb48fa65ea78a399294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c54bea818e4df794095f62233e2984" is_pre_election: true
I20260925 11:05:14.258908 10323 raft_consensus.cc:2454] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55f38c650a854cb48fa65ea78a399294 in term 0.
I20260925 11:05:14.259112 10190 raft_consensus.cc:2454] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55f38c650a854cb48fa65ea78a399294 in term 0.
I20260925 11:05:14.259534 9990 leader_election.cc:302] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55f38c650a854cb48fa65ea78a399294, d5b727e4c16641ceae0c47a86b906e03; no voters:
I20260925 11:05:14.259891 10394 raft_consensus.cc:2781] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:05:14.260015 10394 raft_consensus.cc:482] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:05:14.260118 10394 raft_consensus.cc:3037] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:14.262327 10394 raft_consensus.cc:504] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } }
I20260925 11:05:14.262871 10394 leader_election.cc:288] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [CANDIDATE]: Term 1 election: Requested vote from peers 48c54bea818e4df794095f62233e2984 (127.24.159.194:41791), d5b727e4c16641ceae0c47a86b906e03 (127.24.159.195:45401)
I20260925 11:05:14.263329 10190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a62b542153044f18e188efdc3c8105e" candidate_uuid: "55f38c650a854cb48fa65ea78a399294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48c54bea818e4df794095f62233e2984"
I20260925 11:05:14.263341 10323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a62b542153044f18e188efdc3c8105e" candidate_uuid: "55f38c650a854cb48fa65ea78a399294" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5b727e4c16641ceae0c47a86b906e03"
I20260925 11:05:14.263499 10190 raft_consensus.cc:3037] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:14.263506 10323 raft_consensus.cc:3037] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:14.265741 10323 raft_consensus.cc:2454] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55f38c650a854cb48fa65ea78a399294 in term 1.
I20260925 11:05:14.265810 10190 raft_consensus.cc:2454] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55f38c650a854cb48fa65ea78a399294 in term 1.
I20260925 11:05:14.266148 9990 leader_election.cc:302] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55f38c650a854cb48fa65ea78a399294, d5b727e4c16641ceae0c47a86b906e03; no voters:
I20260925 11:05:14.266450 10394 raft_consensus.cc:2781] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:14.266887 10394 raft_consensus.cc:686] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 1 LEADER]: Becoming Leader. State: Replica: 55f38c650a854cb48fa65ea78a399294, State: Running, Role: LEADER
I20260925 11:05:14.267334 10394 consensus_queue.cc:237] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } }
I20260925 11:05:14.272293 9911 catalog_manager.cc:5495] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55f38c650a854cb48fa65ea78a399294 (127.24.159.193). New cstate: current_term: 1 leader_uuid: "55f38c650a854cb48fa65ea78a399294" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55f38c650a854cb48fa65ea78a399294" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39775 } health_report { overall_health: HEALTHY } } }
I20260925 11:05:14.297531 10385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.292532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.24.159.193:39775)
0925 11:05:14.292710 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.292715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.292732 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:05:14.292820 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.292824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.292910 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.293029 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:05:14.293382 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.294103 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.295580 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.295995 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.296097 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.296531 (+ 434us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:05:14.297045 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:05:14.297178 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.297252 (+ 74us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.297322 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:05:14.305678 10190 raft_consensus.cc:1261] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [term 1 FOLLOWER]: Refusing update from remote peer 55f38c650a854cb48fa65ea78a399294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:14.305678 10323 raft_consensus.cc:1261] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [term 1 FOLLOWER]: Refusing update from remote peer 55f38c650a854cb48fa65ea78a399294: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:14.306630 10397 consensus_queue.cc:1035] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5b727e4c16641ceae0c47a86b906e03" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 45401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:14.306980 10394 consensus_queue.cc:1035] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48c54bea818e4df794095f62233e2984" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:14.317948 10401 mvcc.cc:205] Tried to move back new op lower bound from 7333209351386640384 to 7333209351242723328. Current Snapshot: MvccSnapshot[applied={T|T < 7333209351386640384}]
I20260925 11:05:14.320058 10402 mvcc.cc:205] Tried to move back new op lower bound from 7333209351386640384 to 7333209351242723328. Current Snapshot: MvccSnapshot[applied={T|T < 7333209351386640384}]
I20260925 11:05:14.330385 9911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:14.330650 9911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36072:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:14.333923 9911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 310a75de14ef498a92094f35672b3041: Sending DeleteTablet for 3 replicas of tablet 9a62b542153044f18e188efdc3c8105e
I20260925 11:05:14.335047 10170 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a62b542153044f18e188efdc3c8105e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:14 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:54936
I20260925 11:05:14.335192 10036 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a62b542153044f18e188efdc3c8105e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:14 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:33442
I20260925 11:05:14.336318 10303 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a62b542153044f18e188efdc3c8105e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:14 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:52706
I20260925 11:05:14.337009 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 9973
I20260925 11:05:14.345465 10410 tablet_replica.cc:330] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: stopping tablet replica
I20260925 11:05:14.346078 10410 raft_consensus.cc:2229] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:05:14.346809 10410 raft_consensus.cc:2258] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:14.349169 10410 ts_tablet_manager.cc:1905] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:14.352288 10409 tablet_replica.cc:330] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: stopping tablet replica
I20260925 11:05:14.352736 10409 raft_consensus.cc:2229] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:14.353278 10409 raft_consensus.cc:2258] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:14.355676 10409 ts_tablet_manager.cc:1905] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:14.359187 10410 ts_tablet_manager.cc:1918] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:14.359369 10410 log.cc:1196] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/9a62b542153044f18e188efdc3c8105e
I20260925 11:05:14.359902 10410 ts_tablet_manager.cc:1939] T 9a62b542153044f18e188efdc3c8105e P 55f38c650a854cb48fa65ea78a399294: Deleting consensus metadata
I20260925 11:05:14.361759 9897 catalog_manager.cc:4841] TS 55f38c650a854cb48fa65ea78a399294 (127.24.159.193:39775): tablet 9a62b542153044f18e188efdc3c8105e (table test-table [id=8c3a5b9963c94c4880af5f75e86ea47a]) successfully deleted
I20260925 11:05:14.362692 10411 tablet_replica.cc:330] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: stopping tablet replica
I20260925 11:05:14.363095 10409 ts_tablet_manager.cc:1918] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:14.363185 10411 raft_consensus.cc:2229] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:14.363353 10409 log.cc:1196] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/9a62b542153044f18e188efdc3c8105e
I20260925 11:05:14.363722 10411 raft_consensus.cc:2258] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:14.363957 10409 ts_tablet_manager.cc:1939] T 9a62b542153044f18e188efdc3c8105e P 48c54bea818e4df794095f62233e2984: Deleting consensus metadata
I20260925 11:05:14.366353 9896 catalog_manager.cc:4841] TS 48c54bea818e4df794095f62233e2984 (127.24.159.194:41791): tablet 9a62b542153044f18e188efdc3c8105e (table test-table [id=8c3a5b9963c94c4880af5f75e86ea47a]) successfully deleted
I20260925 11:05:14.366613 10411 ts_tablet_manager.cc:1905] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 25 11:05:14 dist-test-slave-trms krb5kdc[9867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:05:14.373173 10411 ts_tablet_manager.cc:1918] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:14.373334 10411 log.cc:1196] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/9a62b542153044f18e188efdc3c8105e
I20260925 11:05:14.373842 10411 ts_tablet_manager.cc:1939] T 9a62b542153044f18e188efdc3c8105e P d5b727e4c16641ceae0c47a86b906e03: Deleting consensus metadata
I20260925 11:05:14.375789 9897 catalog_manager.cc:4841] TS d5b727e4c16641ceae0c47a86b906e03 (127.24.159.195:45401): tablet 9a62b542153044f18e188efdc3c8105e (table test-table [id=8c3a5b9963c94c4880af5f75e86ea47a]) successfully deleted
I20260925 11:05:14.380218 10385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.338067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.24.159.193:39775)
0925 11:05:14.338300 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.338307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.361347 (+ 23040us) server_negotiation.cc:407] Connection header received
0925 11:05:14.361839 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.361845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.361978 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.362055 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:14.363978 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.364867 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.366881 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.367570 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.367706 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.372910 (+ 5204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:14.372929 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:14.372949 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:14.373004 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:14.376933 (+ 3929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.377624 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.377632 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.377634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.377793 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.378181 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.378185 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.378214 (+ 29us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.378607 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:14.378896 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.379340 (+ 444us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.379844 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:05:14.380818 10096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:14.479969 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 9973
I20260925 11:05:14.495115 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 10107
Sep 25 11:05:14 dist-test-slave-trms krb5kdc[9867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:05:14.509661 10381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.495934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54938 (local address 127.24.159.194:41791)
0925 11:05:14.496119 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.496125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.496157 (+ 32us) server_negotiation.cc:407] Connection header received
0925 11:05:14.496221 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.496226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.496287 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.496381 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:14.497625 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.498399 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.499673 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.500069 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.500182 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.504288 (+ 4106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:14.504305 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:14.504320 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:14.504356 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:14.507032 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.507710 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.507717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.507720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.507844 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.508177 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.508181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.508185 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.508561 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:14.508849 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.509120 (+ 271us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.509335 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:05:14.510368 10230 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:14.603273 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 10107
I20260925 11:05:14.617123 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 10240
Sep 25 11:05:14 dist-test-slave-trms krb5kdc[9867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334312, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:05:14.631551 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:14.617849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52708 (local address 127.24.159.195:45401)
0925 11:05:14.618039 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:14.618044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:14.618062 (+ 18us) server_negotiation.cc:407] Connection header received
0925 11:05:14.618184 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:14.618222 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:14.618300 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:14.618379 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:14.619653 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.620415 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.621699 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:14.622145 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:14.622309 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:14.626399 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:14.626416 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:14.626433 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:14.626473 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:14.628936 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.629637 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.629643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.629646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.629782 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:14.630066 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:14.630069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:14.630070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:14.630573 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:14.630786 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:14.631020 (+ 234us) server_negotiation.cc:299] Negotiation successful
0925 11:05:14.631225 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 11:05:14.632192 10363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:14.720023 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 10240
I20260925 11:05:14.733825 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 9880
I20260925 11:05:14.734915 9942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:14.843161 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 9880
2026-09-25T11:05:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (3997 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-25T11:05:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:14Z Disabled control of system clock
I20260925 11:05:14.915648 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36541
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:36541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 11:05:15.232589 10420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:15.232899 10420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:15.233018 10420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:15.240260 10420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:15.240376 10420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:15.240442 10420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:15.240521 10420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:15.240581 10420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:15.249651 10420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:36541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:15.250926 10420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:15.252674 10420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:15.263857 10428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:15.263830 10426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:15.264055 10425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:15.264672 10420 server_base.cc:1034] running on GCE node
I20260925 11:05:15.265426 10420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:15.266822 10420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:15.268013 10420 hybrid_clock.cc:648] HybridClock initialized: now 1790334315267957 us; error 45 us; skew 500 ppm
I20260925 11:05:15.268329 10420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:15.270437 10420 webserver.cc:460] Webserver started at http://127.24.159.254:39173/ using document root <none> and password file <none>
I20260925 11:05:15.271054 10420 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:15.271147 10420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:15.271399 10420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:15.274495 10420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "a240f72503204801a94fcfb06a591a9b"
format_stamp: "Formatted at 2026-09-25 11:05:15 on dist-test-slave-trms"
server_key: "3e2b0224780d420319280cd0e61dd8fc"
server_key_iv: "1b6b4dbede600a1e7343b7091a5d4ec6"
server_key_version: "encryptionkey@0"
I20260925 11:05:15.275334 10420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "a240f72503204801a94fcfb06a591a9b"
format_stamp: "Formatted at 2026-09-25 11:05:15 on dist-test-slave-trms"
server_key: "3e2b0224780d420319280cd0e61dd8fc"
server_key_iv: "1b6b4dbede600a1e7343b7091a5d4ec6"
server_key_version: "encryptionkey@0"
I20260925 11:05:15.280351 10420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:05:15.283783 10434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:15.285126 10420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:05:15.285270 10420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "a240f72503204801a94fcfb06a591a9b"
format_stamp: "Formatted at 2026-09-25 11:05:15 on dist-test-slave-trms"
server_key: "3e2b0224780d420319280cd0e61dd8fc"
server_key_iv: "1b6b4dbede600a1e7343b7091a5d4ec6"
server_key_version: "encryptionkey@0"
I20260925 11:05:15.285462 10420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:15.304019 10420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:15.304848 10420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:15.305105 10420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:15.321838 10420 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:36541
I20260925 11:05:15.321846 10485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:36541 every 8 connection(s)
I20260925 11:05:15.323235 10420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:15.325896 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 10420
I20260925 11:05:15.326056 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:15.326532 25215 external_mini_cluster.cc:1428] Setting key 1401280e52276829330226facc37f2d6
I20260925 11:05:15.326961 10486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:15.340324 10486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b: Bootstrap starting.
I20260925 11:05:15.340400 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:15.328266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52670 (local address 127.24.159.254:36541)
0925 11:05:15.329233 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:15.329247 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:15.329286 (+ 39us) server_negotiation.cc:407] Connection header received
0925 11:05:15.331353 (+ 2067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:15.331377 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:15.331838 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:15.332161 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:15.332784 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:15.334119 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:15.335901 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:15.336560 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:15.336699 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:15.338029 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:15.338037 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:15.338051 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:15.338686 (+ 635us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:15.338768 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:15.338795 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:15.338890 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:15.339120 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 11:05:15.339438 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":313,"threads_started":1}
I20260925 11:05:15.344039 10486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:15.344995 10486 log.cc:824] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:15.347622 10486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b: No bootstrap required, opened a new log
I20260925 11:05:15.354528 10486 raft_consensus.cc:348] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240f72503204801a94fcfb06a591a9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36541 } }
I20260925 11:05:15.354892 10486 raft_consensus.cc:374] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:15.354980 10486 raft_consensus.cc:729] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a240f72503204801a94fcfb06a591a9b, State: Initialized, Role: FOLLOWER
I20260925 11:05:15.355631 10486 consensus_queue.cc:260] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240f72503204801a94fcfb06a591a9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36541 } }
I20260925 11:05:15.355908 10486 raft_consensus.cc:388] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:15.356047 10486 raft_consensus.cc:482] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:15.356225 10486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:15.358472 10486 raft_consensus.cc:504] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240f72503204801a94fcfb06a591a9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36541 } }
I20260925 11:05:15.358930 10486 leader_election.cc:302] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a240f72503204801a94fcfb06a591a9b; no voters:
I20260925 11:05:15.359445 10486 leader_election.cc:288] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:15.359575 10491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:15.360224 10491 raft_consensus.cc:686] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [term 1 LEADER]: Becoming Leader. State: Replica: a240f72503204801a94fcfb06a591a9b, State: Running, Role: LEADER
I20260925 11:05:15.360703 10491 consensus_queue.cc:237] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240f72503204801a94fcfb06a591a9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36541 } }
I20260925 11:05:15.361411 10486 sys_catalog.cc:564] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:15.363461 10493 sys_catalog.cc:455] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a240f72503204801a94fcfb06a591a9b. Latest consensus state: current_term: 1 leader_uuid: "a240f72503204801a94fcfb06a591a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240f72503204801a94fcfb06a591a9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36541 } } }
I20260925 11:05:15.363914 10493 sys_catalog.cc:458] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:15.363819 10492 sys_catalog.cc:455] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a240f72503204801a94fcfb06a591a9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a240f72503204801a94fcfb06a591a9b" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36541 } } }
I20260925 11:05:15.364110 10492 sys_catalog.cc:458] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:15.366330 10500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:15.370541 10500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:15.379792 10500 catalog_manager.cc:1347] Generated new cluster ID: 040e490f6c1c4bf5b2d4bffa4a84ff2a
I20260925 11:05:15.379909 10500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:15.389616 10500 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:15.391402 10500 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:15.411119 10500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b: Generated new TSK 0
I20260925 11:05:15.412360 10500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:15.421358 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36541
--builtin_ntp_servers=127.24.159.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 11:05:15.740844 10510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:15.741144 10510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:15.741273 10510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:15.748518 10510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:15.748634 10510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:15.748770 10510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:05:15.757828 10510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:15.759115 10510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:15.760847 10510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:15.773829 10515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:15.773793 10516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:15.773761 10518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:15.774559 10510 server_base.cc:1034] running on GCE node
I20260925 11:05:15.775159 10510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:15.776129 10510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:15.777338 10510 hybrid_clock.cc:648] HybridClock initialized: now 1790334315777289 us; error 43 us; skew 500 ppm
I20260925 11:05:15.777647 10510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:15.779946 10510 webserver.cc:460] Webserver started at http://127.24.159.193:42557/ using document root <none> and password file <none>
I20260925 11:05:15.780619 10510 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:15.780720 10510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:15.781009 10510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:15.783989 10510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "c164db5f190d4a5a9c4e0abde9f027b2"
format_stamp: "Formatted at 2026-09-25 11:05:15 on dist-test-slave-trms"
server_key: "44b09e5c02b64b26c2c7658683c57683"
server_key_iv: "b9cbb4a513af41c601134265b7b48e43"
server_key_version: "encryptionkey@0"
I20260925 11:05:15.784760 10510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "c164db5f190d4a5a9c4e0abde9f027b2"
format_stamp: "Formatted at 2026-09-25 11:05:15 on dist-test-slave-trms"
server_key: "44b09e5c02b64b26c2c7658683c57683"
server_key_iv: "b9cbb4a513af41c601134265b7b48e43"
server_key_version: "encryptionkey@0"
I20260925 11:05:15.789870 10510 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:05:15.793354 10524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:15.794736 10510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 11:05:15.794893 10510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "c164db5f190d4a5a9c4e0abde9f027b2"
format_stamp: "Formatted at 2026-09-25 11:05:15 on dist-test-slave-trms"
server_key: "44b09e5c02b64b26c2c7658683c57683"
server_key_iv: "b9cbb4a513af41c601134265b7b48e43"
server_key_version: "encryptionkey@0"
I20260925 11:05:15.795090 10510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:15.809469 10510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:15.810340 10510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:15.810622 10510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:15.811560 10510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:15.813011 10510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:15.813105 10510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:15.813212 10510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:15.813282 10510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:15.843441 10510 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:39683
I20260925 11:05:15.843480 10636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:39683 every 8 connection(s)
I20260925 11:05:15.844763 10510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:05:15.854449 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 10510
I20260925 11:05:15.854624 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:05:15.855159 25215 external_mini_cluster.cc:1428] Setting key 6e9ab476289c610ce8ed4faca9ef5ca9
I20260925 11:05:15.857169 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:15.847258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:36509 (local address 127.24.159.254:36541)
0925 11:05:15.847647 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:15.847651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:15.848965 (+ 1314us) server_negotiation.cc:407] Connection header received
0925 11:05:15.850789 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:15.850794 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:15.850863 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:15.850966 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:15.852005 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:15.852736 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:15.854257 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:15.854882 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:15.855021 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:15.856381 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:15.856387 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:15.856391 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:15.856409 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:15.856446 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:15.856459 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:15.856532 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:15.856797 (+ 265us) server_negotiation.cc:299] Negotiation successful
0925 11:05:15.856897 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":185,"threads_started":1}
I20260925 11:05:15.858079 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36541
--builtin_ntp_servers=127.24.159.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 11:05:15.858475 10640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:15.847876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36541 (local address 127.24.159.193:36509)
0925 11:05:15.848813 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:15.848851 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:15.850514 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:15.851178 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:15.851187 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:15.851282 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:15.851854 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:15.851870 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:15.852851 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:15.852855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:15.854076 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:15.854088 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:15.855081 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:15.855086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:15.855306 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:15.856020 (+ 714us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:05:15.856023 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:15.856108 (+ 85us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:05:15.856198 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:15.856546 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:15.856558 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:15.856566 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 11:05:15.856736 (+ 170us) client_negotiation.cc:240] Negotiation successful
0925 11:05:15.857111 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":264,"threads_started":1}
I20260925 11:05:15.859980 10637 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36541
I20260925 11:05:15.860301 10637 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:15.861191 10637 heartbeater.cc:511] Master 127.24.159.254:36541 requested a full tablet report, sending...
I20260925 11:05:15.863523 10451 ts_manager.cc:194] Registered new tserver with Master: c164db5f190d4a5a9c4e0abde9f027b2 (127.24.159.193:39683)
I20260925 11:05:15.865072 10451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.193:36509
W20260925 11:05:16.180248 10641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:16.180544 10641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:16.180657 10641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:16.187879 10641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:16.187991 10641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:16.188144 10641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:05:16.197302 10641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:16.198606 10641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:16.200373 10641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:16.212927 10649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:16.212996 10647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:16.212970 10646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:16.213735 10641 server_base.cc:1034] running on GCE node
I20260925 11:05:16.214358 10641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:16.215281 10641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:16.216499 10641 hybrid_clock.cc:648] HybridClock initialized: now 1790334316216443 us; error 47 us; skew 500 ppm
I20260925 11:05:16.216809 10641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:16.222012 10641 webserver.cc:460] Webserver started at http://127.24.159.194:42621/ using document root <none> and password file <none>
I20260925 11:05:16.222769 10641 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:16.222872 10641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:16.223167 10641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:16.225991 10641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "394b6bd108e84732865db7759de58b88"
format_stamp: "Formatted at 2026-09-25 11:05:16 on dist-test-slave-trms"
server_key: "996419b6d203098f8add113b0a0ed449"
server_key_iv: "84e817c8953f1fcbc62f788882400e24"
server_key_version: "encryptionkey@0"
I20260925 11:05:16.226816 10641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "394b6bd108e84732865db7759de58b88"
format_stamp: "Formatted at 2026-09-25 11:05:16 on dist-test-slave-trms"
server_key: "996419b6d203098f8add113b0a0ed449"
server_key_iv: "84e817c8953f1fcbc62f788882400e24"
server_key_version: "encryptionkey@0"
I20260925 11:05:16.231907 10641 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:05:16.235342 10655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:16.236680 10641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260925 11:05:16.236826 10641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "394b6bd108e84732865db7759de58b88"
format_stamp: "Formatted at 2026-09-25 11:05:16 on dist-test-slave-trms"
server_key: "996419b6d203098f8add113b0a0ed449"
server_key_iv: "84e817c8953f1fcbc62f788882400e24"
server_key_version: "encryptionkey@0"
I20260925 11:05:16.237017 10641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:16.256704 10641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:16.257542 10641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:16.257833 10641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:16.258872 10641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:16.260437 10641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:16.260533 10641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:16.260636 10641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:16.260705 10641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:16.299296 10641 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:33257
I20260925 11:05:16.299296 10767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:33257 every 8 connection(s)
I20260925 11:05:16.301033 10641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:05:16.302446 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 10641
I20260925 11:05:16.302642 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:05:16.303196 25215 external_mini_cluster.cc:1428] Setting key b34e339cf82923a5a0f73b112024fe63
I20260925 11:05:16.307030 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36541
--builtin_ntp_servers=127.24.159.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 11:05:16.321895 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.305520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:49733 (local address 127.24.159.254:36541)
0925 11:05:16.306479 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:16.306486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:16.306509 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:05:16.314329 (+ 7820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:16.314336 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:16.314440 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:16.314557 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:16.316099 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.317208 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.318883 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.319536 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.319656 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.321001 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:16.321033 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:16.321038 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:16.321062 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:16.321101 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:16.321117 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:16.321194 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:16.321502 (+ 308us) server_negotiation.cc:299] Negotiation successful
0925 11:05:16.321605 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":827}
I20260925 11:05:16.324194 10770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.304970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36541 (local address 127.24.159.194:49733)
0925 11:05:16.306146 (+ 1176us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:16.306236 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:16.308629 (+ 2393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:16.314916 (+ 6287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:16.314929 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:16.315102 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:16.315919 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.315946 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.317371 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.317377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.318744 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.318757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.319746 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.319753 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.319980 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.320660 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:05:16.320663 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:16.320703 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:05:16.320795 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:16.321254 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:16.321265 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:16.321273 (+ 8us) client_negotiation.cc:769] Sending connection context
0925 11:05:16.322514 (+ 1241us) client_negotiation.cc:240] Negotiation successful
0925 11:05:16.322939 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":344,"threads_started":1}
I20260925 11:05:16.325975 10768 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36541
I20260925 11:05:16.326356 10768 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:16.327275 10768 heartbeater.cc:511] Master 127.24.159.254:36541 requested a full tablet report, sending...
I20260925 11:05:16.328755 10451 ts_manager.cc:194] Registered new tserver with Master: 394b6bd108e84732865db7759de58b88 (127.24.159.194:33257)
I20260925 11:05:16.329607 10451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.194:49733
W20260925 11:05:16.633899 10771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:16.634179 10771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:16.634331 10771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:16.641697 10771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:16.641798 10771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:16.641940 10771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:05:16.651150 10771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:36541
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:16.652415 10771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:16.654166 10771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:16.666750 10776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:16.666766 10777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:16.666920 10779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:16.667356 10771 server_base.cc:1034] running on GCE node
I20260925 11:05:16.668110 10771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:16.669312 10771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:16.670538 10771 hybrid_clock.cc:648] HybridClock initialized: now 1790334316670476 us; error 55 us; skew 500 ppm
I20260925 11:05:16.670876 10771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:16.673151 10771 webserver.cc:460] Webserver started at http://127.24.159.195:42495/ using document root <none> and password file <none>
I20260925 11:05:16.673846 10771 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:16.673949 10771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:16.674305 10771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:16.677549 10771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a"
format_stamp: "Formatted at 2026-09-25 11:05:16 on dist-test-slave-trms"
server_key: "6e725a9096366a6fc07884f4866d7829"
server_key_iv: "2b7aa299b693db7595e7bdd9d4853bfb"
server_key_version: "encryptionkey@0"
I20260925 11:05:16.678393 10771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a"
format_stamp: "Formatted at 2026-09-25 11:05:16 on dist-test-slave-trms"
server_key: "6e725a9096366a6fc07884f4866d7829"
server_key_iv: "2b7aa299b693db7595e7bdd9d4853bfb"
server_key_version: "encryptionkey@0"
I20260925 11:05:16.683684 10771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260925 11:05:16.687099 10785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:16.688432 10771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:05:16.688584 10771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a"
format_stamp: "Formatted at 2026-09-25 11:05:16 on dist-test-slave-trms"
server_key: "6e725a9096366a6fc07884f4866d7829"
server_key_iv: "2b7aa299b693db7595e7bdd9d4853bfb"
server_key_version: "encryptionkey@0"
I20260925 11:05:16.688779 10771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:16.704885 10771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:16.705705 10771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:16.705986 10771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:16.707003 10771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:16.708431 10771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:16.708529 10771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:16.708634 10771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:16.708704 10771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:16.738246 10771 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:44865
I20260925 11:05:16.738309 10897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:44865 every 8 connection(s)
I20260925 11:05:16.739516 10771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:05:16.746318 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 10771
I20260925 11:05:16.746500 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:05:16.746939 25215 external_mini_cluster.cc:1428] Setting key 445870babc1c4045ea52aedeac475203
I20260925 11:05:16.751811 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.741998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:34217 (local address 127.24.159.254:36541)
0925 11:05:16.742174 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:16.742179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:16.743594 (+ 1415us) server_negotiation.cc:407] Connection header received
0925 11:05:16.745293 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:16.745298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:16.745366 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:16.745454 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:16.746686 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.747511 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.749395 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.749863 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.749956 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.751188 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:16.751192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:16.751195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:16.751211 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:16.751240 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:16.751251 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:16.751316 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:16.751500 (+ 184us) server_negotiation.cc:299] Negotiation successful
0925 11:05:16.751573 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:05:16.752681 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.742546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:36541 (local address 127.24.159.195:34217)
0925 11:05:16.743436 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:16.743471 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:16.745065 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:16.745646 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:16.745654 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:16.745742 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:16.746508 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.746533 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.747691 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.747696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.749273 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.749288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.750041 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.750046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.750302 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.750934 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:05:16.750937 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:16.750975 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:05:16.751048 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:16.751321 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:16.751327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:16.751333 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 11:05:16.751451 (+ 118us) client_negotiation.cc:240] Negotiation successful
0925 11:05:16.751731 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":260,"threads_started":1}
I20260925 11:05:16.753803 10898 heartbeater.cc:347] Connected to a master server at 127.24.159.254:36541
I20260925 11:05:16.754060 10898 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:16.754786 10898 heartbeater.cc:511] Master 127.24.159.254:36541 requested a full tablet report, sending...
I20260925 11:05:16.756093 10451 ts_manager.cc:194] Registered new tserver with Master: ca63b64221dc4ea1bc5ba1dd924fcd2a (127.24.159.195:44865)
I20260925 11:05:16.756892 10451 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.195:34217
I20260925 11:05:16.761931 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:05:16.771741 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.765968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.24.159.254:36541)
0925 11:05:16.766168 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:16.766175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:16.766558 (+ 383us) server_negotiation.cc:407] Connection header received
0925 11:05:16.766726 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:16.766732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:16.766797 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:16.766876 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:16.767355 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.768063 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.769393 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.769929 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.770049 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.771056 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:16.771062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:16.771066 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:16.771083 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:16.771122 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:16.771139 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:16.771215 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:16.771344 (+ 129us) server_negotiation.cc:299] Negotiation successful
0925 11:05:16.771438 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:05:16.775400 10451 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:05:16.782073 10451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:05:16.814785 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.794090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47808 (local address 127.24.159.195:44865)
0925 11:05:16.805817 (+ 11727us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:16.805827 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:16.806958 (+ 1131us) server_negotiation.cc:407] Connection header received
0925 11:05:16.807070 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:16.807078 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:16.807366 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:16.807556 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:16.808417 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.810030 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.812262 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.813380 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.813571 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.813795 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:16.814268 (+ 473us) server_negotiation.cc:299] Negotiation successful
0925 11:05:16.814430 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11550,"thread_start_us":11386,"threads_started":1}
I20260925 11:05:16.814769 10910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.793885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:44865 (local address 127.0.0.1:47808)
0925 11:05:16.806448 (+ 12563us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:16.806508 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:16.806720 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:16.807751 (+ 1031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:16.807759 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:16.807790 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:16.808197 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.808216 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.810218 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.810223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.812065 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.812078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.813898 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.813902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.814029 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.814063 (+ 34us) client_negotiation.cc:769] Sending connection context
0925 11:05:16.814180 (+ 117us) client_negotiation.cc:240] Negotiation successful
0925 11:05:16.814403 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12262,"mutex_wait_us":1,"thread_start_us":11267,"threads_started":1}
I20260925 11:05:16.816092 10914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.794008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39683 (local address 127.0.0.1:60792)
0925 11:05:16.807123 (+ 13115us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:16.807144 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:16.807254 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:16.808040 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:16.808044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:16.808066 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:16.808450 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.808464 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.810430 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.810435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.812751 (+ 2316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.812766 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.815367 (+ 2601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.815373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.815548 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.815579 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 11:05:16.815660 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 11:05:16.815785 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12982,"mutex_wait_us":1,"thread_start_us":13448,"threads_started":1}
I20260925 11:05:16.816437 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.796760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.24.159.194:33257)
0925 11:05:16.806713 (+ 9953us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:16.806721 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:16.807820 (+ 1099us) server_negotiation.cc:407] Connection header received
0925 11:05:16.808077 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:16.808085 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:16.808299 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:16.808493 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:16.809078 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.810523 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.813000 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.814419 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.814603 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.814811 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:16.815920 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0925 11:05:16.816090 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9795,"thread_start_us":9679,"threads_started":1}
I20260925 11:05:16.815822 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.796163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:33257 (local address 127.0.0.1:38722)
0925 11:05:16.807711 (+ 11548us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:16.807732 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:16.807912 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:16.808487 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:16.808491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:16.808523 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:16.808862 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.808874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.810660 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.810665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.812808 (+ 2143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.812817 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.815028 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.815035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.815207 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.815237 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 11:05:16.815335 (+ 98us) client_negotiation.cc:240] Negotiation successful
0925 11:05:16.815472 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11434,"mutex_wait_us":24,"thread_start_us":10693,"threads_started":1}
I20260925 11:05:16.818421 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.794297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60792 (local address 127.24.159.193:39683)
0925 11:05:16.805054 (+ 10757us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:16.805064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:16.807259 (+ 2195us) server_negotiation.cc:407] Connection header received
0925 11:05:16.807401 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:16.807410 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:16.807655 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:16.807847 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:16.808658 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.810304 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.813000 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.814728 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.815037 (+ 309us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.815240 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:16.815735 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 11:05:16.815900 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10565,"thread_start_us":10426,"threads_started":1}
I20260925 11:05:16.820343 10703 tablet_service.cc:1467] Processing CreateTablet for tablet da032ea2f63e4a19a99cceb6c4be32ac (DEFAULT_TABLE table=test-table [id=dd1099cf319643b693dfad71a549c064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:16.821900 10703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da032ea2f63e4a19a99cceb6c4be32ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:16.821671 10572 tablet_service.cc:1467] Processing CreateTablet for tablet da032ea2f63e4a19a99cceb6c4be32ac (DEFAULT_TABLE table=test-table [id=dd1099cf319643b693dfad71a549c064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:16.822726 10833 tablet_service.cc:1467] Processing CreateTablet for tablet da032ea2f63e4a19a99cceb6c4be32ac (DEFAULT_TABLE table=test-table [id=dd1099cf319643b693dfad71a549c064]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:16.824156 10833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da032ea2f63e4a19a99cceb6c4be32ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:16.826601 10572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet da032ea2f63e4a19a99cceb6c4be32ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:16.836730 10916 tablet_bootstrap.cc:492] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: Bootstrap starting.
I20260925 11:05:16.838711 10918 tablet_bootstrap.cc:492] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: Bootstrap starting.
I20260925 11:05:16.840756 10916 tablet_bootstrap.cc:654] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:16.841503 10918 tablet_bootstrap.cc:654] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:16.842142 10916 log.cc:824] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:16.842531 10918 log.cc:824] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:16.844646 10917 tablet_bootstrap.cc:492] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2: Bootstrap starting.
I20260925 11:05:16.847229 10916 tablet_bootstrap.cc:492] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: No bootstrap required, opened a new log
I20260925 11:05:16.847520 10916 ts_tablet_manager.cc:1397] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260925 11:05:16.848433 10917 tablet_bootstrap.cc:654] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:16.849808 10917 log.cc:824] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:16.851634 10918 tablet_bootstrap.cc:492] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: No bootstrap required, opened a new log
I20260925 11:05:16.851894 10918 ts_tablet_manager.cc:1397] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20260925 11:05:16.860040 10918 raft_consensus.cc:348] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } }
I20260925 11:05:16.860010 10916 raft_consensus.cc:348] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } }
I20260925 11:05:16.860396 10918 raft_consensus.cc:374] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:16.860482 10918 raft_consensus.cc:729] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca63b64221dc4ea1bc5ba1dd924fcd2a, State: Initialized, Role: FOLLOWER
I20260925 11:05:16.860498 10916 raft_consensus.cc:374] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:16.860605 10916 raft_consensus.cc:729] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394b6bd108e84732865db7759de58b88, State: Initialized, Role: FOLLOWER
I20260925 11:05:16.861290 10918 consensus_queue.cc:260] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } }
I20260925 11:05:16.861306 10916 consensus_queue.cc:260] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } }
I20260925 11:05:16.862983 10768 heartbeater.cc:503] Master 127.24.159.254:36541 was elected leader, sending a full tablet report...
I20260925 11:05:16.863787 10916 ts_tablet_manager.cc:1428] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: Time spent starting tablet: real 0.016s user 0.013s sys 0.002s
I20260925 11:05:16.864861 10898 heartbeater.cc:503] Master 127.24.159.254:36541 was elected leader, sending a full tablet report...
I20260925 11:05:16.864969 10917 tablet_bootstrap.cc:492] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2: No bootstrap required, opened a new log
I20260925 11:05:16.865264 10917 ts_tablet_manager.cc:1397] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2: Time spent bootstrapping tablet: real 0.021s user 0.004s sys 0.016s
I20260925 11:05:16.866245 10918 ts_tablet_manager.cc:1428] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260925 11:05:16.868467 10637 heartbeater.cc:503] Master 127.24.159.254:36541 was elected leader, sending a full tablet report...
I20260925 11:05:16.874557 10917 raft_consensus.cc:348] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } }
I20260925 11:05:16.874957 10917 raft_consensus.cc:374] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:16.875053 10917 raft_consensus.cc:729] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c164db5f190d4a5a9c4e0abde9f027b2, State: Initialized, Role: FOLLOWER
I20260925 11:05:16.875610 10917 consensus_queue.cc:260] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } }
I20260925 11:05:16.876997 10917 ts_tablet_manager.cc:1428] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2: Time spent starting tablet: real 0.012s user 0.007s sys 0.003s
I20260925 11:05:16.947733 10922 raft_consensus.cc:482] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:05:16.948097 10922 raft_consensus.cc:504] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } }
I20260925 11:05:16.949766 10922 leader_election.cc:288] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c164db5f190d4a5a9c4e0abde9f027b2 (127.24.159.193:39683), 394b6bd108e84732865db7759de58b88 (127.24.159.194:33257)
I20260925 11:05:16.955267 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.950320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:60655 (local address 127.24.159.194:33257)
0925 11:05:16.950513 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:16.950519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:16.950607 (+ 88us) server_negotiation.cc:407] Connection header received
0925 11:05:16.950826 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:16.950830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:16.950892 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:16.950966 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:16.951374 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.952070 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.953694 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.954490 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.954628 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.954695 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:16.954920 (+ 225us) server_negotiation.cc:299] Negotiation successful
0925 11:05:16.955037 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:05:16.955325 10900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.950122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:33257 (local address 127.24.159.195:60655)
0925 11:05:16.950513 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:16.950532 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:16.950669 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:16.951049 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:16.951054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:16.951083 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:16.951231 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.951239 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.952196 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.952200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.953538 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.953550 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.954627 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.954633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.954778 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.954803 (+ 25us) client_negotiation.cc:769] Sending connection context
0925 11:05:16.954880 (+ 77us) client_negotiation.cc:240] Negotiation successful
0925 11:05:16.955011 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":21}
I20260925 11:05:16.955823 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.950166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:48847 (local address 127.24.159.193:39683)
0925 11:05:16.950391 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:16.950397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:16.951019 (+ 622us) server_negotiation.cc:407] Connection header received
0925 11:05:16.951196 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:16.951201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:16.951271 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:16.951343 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:16.951870 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.952543 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.954234 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.954980 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.955121 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.955188 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:16.955476 (+ 288us) server_negotiation.cc:299] Negotiation successful
0925 11:05:16.955577 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260925 11:05:16.956318 10925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:16.950166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39683 (local address 127.24.159.195:48847)
0925 11:05:16.950911 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:16.950925 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:16.951057 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:16.951379 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:16.951383 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:16.951405 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:16.951713 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.951721 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.952695 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.952700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.954032 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:16.954044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:16.955135 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:16.955142 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:16.955299 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:16.955325 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 11:05:16.956000 (+ 675us) client_negotiation.cc:240] Negotiation successful
0925 11:05:16.956091 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":4,"thread_start_us":385,"threads_started":1}
I20260925 11:05:16.956430 10723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da032ea2f63e4a19a99cceb6c4be32ac" candidate_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394b6bd108e84732865db7759de58b88" is_pre_election: true
I20260925 11:05:16.956982 10723 raft_consensus.cc:2454] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca63b64221dc4ea1bc5ba1dd924fcd2a in term 0.
I20260925 11:05:16.957322 10592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da032ea2f63e4a19a99cceb6c4be32ac" candidate_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" is_pre_election: true
I20260925 11:05:16.957713 10787 leader_election.cc:302] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394b6bd108e84732865db7759de58b88, ca63b64221dc4ea1bc5ba1dd924fcd2a; no voters:
I20260925 11:05:16.957891 10592 raft_consensus.cc:2454] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca63b64221dc4ea1bc5ba1dd924fcd2a in term 0.
I20260925 11:05:16.958120 10922 raft_consensus.cc:2781] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:05:16.958276 10922 raft_consensus.cc:482] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:05:16.958393 10922 raft_consensus.cc:3037] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:16.960537 10922 raft_consensus.cc:504] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } }
I20260925 11:05:16.961091 10922 leader_election.cc:288] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [CANDIDATE]: Term 1 election: Requested vote from peers c164db5f190d4a5a9c4e0abde9f027b2 (127.24.159.193:39683), 394b6bd108e84732865db7759de58b88 (127.24.159.194:33257)
I20260925 11:05:16.961475 10592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da032ea2f63e4a19a99cceb6c4be32ac" candidate_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c164db5f190d4a5a9c4e0abde9f027b2"
I20260925 11:05:16.961542 10723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "da032ea2f63e4a19a99cceb6c4be32ac" candidate_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "394b6bd108e84732865db7759de58b88"
I20260925 11:05:16.961658 10592 raft_consensus.cc:3037] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:16.961747 10723 raft_consensus.cc:3037] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:16.964099 10723 raft_consensus.cc:2454] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca63b64221dc4ea1bc5ba1dd924fcd2a in term 1.
I20260925 11:05:16.964133 10592 raft_consensus.cc:2454] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca63b64221dc4ea1bc5ba1dd924fcd2a in term 1.
I20260925 11:05:16.964481 10787 leader_election.cc:302] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394b6bd108e84732865db7759de58b88, ca63b64221dc4ea1bc5ba1dd924fcd2a; no voters:
I20260925 11:05:16.964769 10922 raft_consensus.cc:2781] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:16.965176 10922 raft_consensus.cc:686] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 1 LEADER]: Becoming Leader. State: Replica: ca63b64221dc4ea1bc5ba1dd924fcd2a, State: Running, Role: LEADER
I20260925 11:05:16.965657 10922 consensus_queue.cc:237] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } }
I20260925 11:05:16.970362 10451 catalog_manager.cc:5495] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a reported cstate change: term changed from 0 to 1, leader changed from <none> to ca63b64221dc4ea1bc5ba1dd924fcd2a (127.24.159.195). New cstate: current_term: 1 leader_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca63b64221dc4ea1bc5ba1dd924fcd2a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 44865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 } health_report { overall_health: UNKNOWN } } }
W20260925 11:05:16.991178 10899 tablet.cc:2369] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:05:17.006268 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.000782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47822 (local address 127.24.159.195:44865)
0925 11:05:17.000985 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:17.000992 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:17.001087 (+ 95us) server_negotiation.cc:407] Connection header received
0925 11:05:17.001314 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:17.001318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:17.001398 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:17.001533 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:05:17.001917 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:17.002673 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:17.004132 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:17.004557 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:17.004667 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:17.005147 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:05:17.005665 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:05:17.005812 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:17.005911 (+ 99us) server_negotiation.cc:299] Negotiation successful
0925 11:05:17.005993 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:05:17.013968 10592 raft_consensus.cc:1261] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [term 1 FOLLOWER]: Refusing update from remote peer ca63b64221dc4ea1bc5ba1dd924fcd2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:17.013968 10723 raft_consensus.cc:1261] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [term 1 FOLLOWER]: Refusing update from remote peer ca63b64221dc4ea1bc5ba1dd924fcd2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:17.014938 10926 consensus_queue.cc:1035] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c164db5f190d4a5a9c4e0abde9f027b2" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 39683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:17.015275 10922 consensus_queue.cc:1035] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "394b6bd108e84732865db7759de58b88" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 33257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:17.026118 10929 mvcc.cc:205] Tried to move back new op lower bound from 7333209362480025600 to 7333209362295001088. Current Snapshot: MvccSnapshot[applied={T|T < 7333209362480025600}]
I20260925 11:05:17.039075 10451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:17.039337 10451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:17.042821 10451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a240f72503204801a94fcfb06a591a9b: Sending DeleteTablet for 3 replicas of tablet da032ea2f63e4a19a99cceb6c4be32ac
I20260925 11:05:17.044003 10833 tablet_service.cc:1514] Processing DeleteTablet for tablet da032ea2f63e4a19a99cceb6c4be32ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:17 UTC) from {username='slave'} at 127.0.0.1:47808
I20260925 11:05:17.044034 10572 tablet_service.cc:1514] Processing DeleteTablet for tablet da032ea2f63e4a19a99cceb6c4be32ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:17 UTC) from {username='slave'} at 127.0.0.1:60792
I20260925 11:05:17.044360 10703 tablet_service.cc:1514] Processing DeleteTablet for tablet da032ea2f63e4a19a99cceb6c4be32ac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:17 UTC) from {username='slave'} at 127.0.0.1:38722
I20260925 11:05:17.045544 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 10510
I20260925 11:05:17.057509 10938 tablet_replica.cc:330] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2: stopping tablet replica
I20260925 11:05:17.058013 10938 raft_consensus.cc:2229] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:17.058866 10940 tablet_replica.cc:330] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: stopping tablet replica
I20260925 11:05:17.059285 10940 raft_consensus.cc:2229] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:17.060467 10938 raft_consensus.cc:2258] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:17.059825 10940 raft_consensus.cc:2258] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:17.062825 10938 ts_tablet_manager.cc:1905] T da032ea2f63e4a19a99cceb6c4be32ac P c164db5f190d4a5a9c4e0abde9f027b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:17.064249 10939 tablet_replica.cc:330] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: stopping tablet replica
I20260925 11:05:17.064673 10939 raft_consensus.cc:2229] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:05:17.065546 10939 raft_consensus.cc:2258] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:17.066063 10940 ts_tablet_manager.cc:1905] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:17.068091 10939 ts_tablet_manager.cc:1905] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:17.068343 10913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.046776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60800 (local address 127.24.159.193:39683)
0925 11:05:17.046968 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:17.046974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:17.058253 (+ 11279us) server_negotiation.cc:407] Connection header received
0925 11:05:17.058765 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:17.058772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:17.058920 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:17.059010 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:17.059425 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:17.060684 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:17.064212 (+ 3528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:17.064832 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:17.064932 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:17.066675 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:17.066689 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:17.066705 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:17.067436 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:17.067506 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:17.067534 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:17.067638 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:17.067891 (+ 253us) server_negotiation.cc:299] Negotiation successful
0925 11:05:17.068009 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:05:17.069767 10632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:17.073338 10940 ts_tablet_manager.cc:1918] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:17.073522 10940 log.cc:1196] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/da032ea2f63e4a19a99cceb6c4be32ac
I20260925 11:05:17.074076 10940 ts_tablet_manager.cc:1939] T da032ea2f63e4a19a99cceb6c4be32ac P 394b6bd108e84732865db7759de58b88: Deleting consensus metadata
I20260925 11:05:17.076179 10436 catalog_manager.cc:4841] TS 394b6bd108e84732865db7759de58b88 (127.24.159.194:33257): tablet da032ea2f63e4a19a99cceb6c4be32ac (table test-table [id=dd1099cf319643b693dfad71a549c064]) successfully deleted
I20260925 11:05:17.077288 10939 ts_tablet_manager.cc:1918] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:17.077489 10939 log.cc:1196] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/da032ea2f63e4a19a99cceb6c4be32ac
I20260925 11:05:17.078079 10939 ts_tablet_manager.cc:1939] T da032ea2f63e4a19a99cceb6c4be32ac P ca63b64221dc4ea1bc5ba1dd924fcd2a: Deleting consensus metadata
I20260925 11:05:17.080004 10438 catalog_manager.cc:4841] TS ca63b64221dc4ea1bc5ba1dd924fcd2a (127.24.159.195:44865): tablet da032ea2f63e4a19a99cceb6c4be32ac (table test-table [id=dd1099cf319643b693dfad71a549c064]) successfully deleted
I20260925 11:05:17.194761 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 10510
I20260925 11:05:17.208674 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 10641
W20260925 11:05:17.208730 10435 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.159.193:39683 (error 108)
I20260925 11:05:17.211124 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.210665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39683 (local address 127.0.0.1:60806)
0925 11:05:17.210824 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:17.210949 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 11:05:17.211900 10435 catalog_manager.cc:4584] TS c164db5f190d4a5a9c4e0abde9f027b2 (127.24.159.193:39683): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet da032ea2f63e4a19a99cceb6c4be32ac: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39683: connect: Connection refused (error 111)
I20260925 11:05:17.216621 10915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.209932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.24.159.194:33257)
0925 11:05:17.210104 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:17.210108 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:17.210125 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:05:17.210182 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:17.210185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:17.210318 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:17.210402 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:17.210867 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:17.211707 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:17.213560 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:17.213968 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:17.214068 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:17.215351 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:17.215360 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:17.215373 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:17.215938 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:17.215987 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:17.216007 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:17.216090 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:17.216228 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 11:05:17.216315 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:05:17.217312 10763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:17.265551 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.264978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39683 (local address 127.0.0.1:60816)
0925 11:05:17.265211 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:17.265329 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260925 11:05:17.266762 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.266344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39683 (local address 127.0.0.1:60832)
0925 11:05:17.266503 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:17.266603 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:05:17.305958 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 10641
I20260925 11:05:17.326547 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 10771
I20260925 11:05:17.328437 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.327941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39683 (local address 127.0.0.1:60834)
0925 11:05:17.328165 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:17.328266 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 11:05:17.329526 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.329098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39683 (local address 127.0.0.1:60838)
0925 11:05:17.329317 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:17.329388 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260925 11:05:17.336536 10912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.328299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.24.159.195:44865)
0925 11:05:17.328473 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:17.328479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:17.328500 (+ 21us) server_negotiation.cc:407] Connection header received
0925 11:05:17.328751 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:17.328757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:17.328868 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:17.328969 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:17.329815 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:17.331047 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:17.332915 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:17.333595 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:17.333716 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:17.334912 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:17.334923 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:17.334943 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:17.335724 (+ 781us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:17.335787 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:17.335814 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:17.335924 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:17.336076 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 11:05:17.336185 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:05:17.337250 10893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:17.410120 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.409505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39683 (local address 127.0.0.1:60852)
0925 11:05:17.409783 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:17.409907 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260925 11:05:17.411312 10911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:17.410847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:39683 (local address 127.0.0.1:60856)
0925 11:05:17.411070 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:17.411174 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:39683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260925 11:05:17.457576 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 10771
I20260925 11:05:17.473852 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 10420
I20260925 11:05:17.474911 10481 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:17.581346 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 10420
2026-09-25T11:05:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2734 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:17 dist-test-slave-trms krb5kdc[10949](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:17 dist-test-slave-trms krb5kdc[10949](info): set up 2 sockets
Sep 25 11:05:17 dist-test-slave-trms krb5kdc[10949](info): commencing operation
krb5kdc: starting...
W20260925 11:05:18.610267 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:18 dist-test-slave-trms krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:05:18.839439 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:41769
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:41769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:19.157923 10962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:19.158248 10962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:19.158378 10962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:19.165548 10962 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:19.165676 10962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:19.165756 10962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:19.165830 10962 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:19.165890 10962 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:19.175712 10962 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:41769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:41769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:19.177031 10962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:19.178368 10962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:19.189821 10967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:19.189774 10968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:19.189791 10970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:19.190420 10962 server_base.cc:1034] running on GCE node
I20260925 11:05:19.191277 10962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:19.192725 10962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:19.193951 10962 hybrid_clock.cc:648] HybridClock initialized: now 1790334319193898 us; error 36 us; skew 500 ppm
I20260925 11:05:19.194351 10962 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:19 dist-test-slave-trms krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:19.197322 10962 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:05:19.198738 10962 webserver.cc:460] Webserver started at http://127.24.159.254:36999/ using document root <none> and password file <none>
I20260925 11:05:19.199395 10962 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:19.199509 10962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:19.199790 10962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:19.202732 10962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "0a7e3a94b312477ea46396345b320145"
format_stamp: "Formatted at 2026-09-25 11:05:19 on dist-test-slave-trms"
server_key: "50f2b199964070777957c410a9a8abe6"
server_key_iv: "bc68cdee389a0477e1ca5f35c6b0fe12"
server_key_version: "encryptionkey@0"
I20260925 11:05:19.203471 10962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "0a7e3a94b312477ea46396345b320145"
format_stamp: "Formatted at 2026-09-25 11:05:19 on dist-test-slave-trms"
server_key: "50f2b199964070777957c410a9a8abe6"
server_key_iv: "bc68cdee389a0477e1ca5f35c6b0fe12"
server_key_version: "encryptionkey@0"
I20260925 11:05:19.208413 10962 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260925 11:05:19.211696 10977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:19.212898 10962 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:05:19.213049 10962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "0a7e3a94b312477ea46396345b320145"
format_stamp: "Formatted at 2026-09-25 11:05:19 on dist-test-slave-trms"
server_key: "50f2b199964070777957c410a9a8abe6"
server_key_iv: "bc68cdee389a0477e1ca5f35c6b0fe12"
server_key_version: "encryptionkey@0"
I20260925 11:05:19.213234 10962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:19.232612 10962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:19.236833 10962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:19.237090 10962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:19.253870 10962 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:41769
I20260925 11:05:19.253892 11028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:41769 every 8 connection(s)
I20260925 11:05:19.255250 10962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:19.258908 11029 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:19.261229 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 10962
I20260925 11:05:19.261391 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:19.261822 25215 external_mini_cluster.cc:1428] Setting key 7ad89bb3bc6a5a5d537dee3a838281cc
I20260925 11:05:19.270279 11029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145: Bootstrap starting.
I20260925 11:05:19.273902 11029 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:19.275216 11029 log.cc:824] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145: Log is configured to *not* fsync() on all Append() calls
Sep 25 11:05:19 dist-test-slave-trms krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:19.278314 11029 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145: No bootstrap required, opened a new log
I20260925 11:05:19.285916 11032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:19.263711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.24.159.254:41769)
0925 11:05:19.264458 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:19.264469 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:19.264509 (+ 40us) server_negotiation.cc:407] Connection header received
0925 11:05:19.266139 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:19.266162 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:19.266603 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:19.266931 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:19.268673 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:19.269961 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:19.271726 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:19.272413 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:19.272536 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:19.277434 (+ 4898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:19.277445 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:19.277465 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:19.277506 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:19.281451 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:19.282175 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:19.282184 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:19.282220 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:19.282423 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:19.282774 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:19.282779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:19.282783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:19.283487 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:19.283798 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:19.284079 (+ 281us) server_negotiation.cc:299] Negotiation successful
0925 11:05:19.284521 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":343,"threads_started":1}
I20260925 11:05:19.288453 11029 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7e3a94b312477ea46396345b320145" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41769 } }
I20260925 11:05:19.288865 11029 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:19.288972 11029 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a7e3a94b312477ea46396345b320145, State: Initialized, Role: FOLLOWER
I20260925 11:05:19.289536 11029 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7e3a94b312477ea46396345b320145" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41769 } }
I20260925 11:05:19.289747 11029 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:19.289871 11029 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:19.290005 11029 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:19.292141 11029 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7e3a94b312477ea46396345b320145" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41769 } }
I20260925 11:05:19.292606 11029 leader_election.cc:302] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a7e3a94b312477ea46396345b320145; no voters:
I20260925 11:05:19.293156 11029 leader_election.cc:288] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:19.293286 11034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:19.293932 11034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [term 1 LEADER]: Becoming Leader. State: Replica: 0a7e3a94b312477ea46396345b320145, State: Running, Role: LEADER
I20260925 11:05:19.294528 11034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7e3a94b312477ea46396345b320145" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41769 } }
I20260925 11:05:19.295215 11029 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:19.297498 11035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a7e3a94b312477ea46396345b320145" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7e3a94b312477ea46396345b320145" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41769 } } }
I20260925 11:05:19.297510 11036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a7e3a94b312477ea46396345b320145. Latest consensus state: current_term: 1 leader_uuid: "0a7e3a94b312477ea46396345b320145" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a7e3a94b312477ea46396345b320145" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41769 } } }
I20260925 11:05:19.297958 11036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:19.297959 11035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:19.302176 11040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:19.306751 11040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:19.315186 11040 catalog_manager.cc:1347] Generated new cluster ID: 45bfd128208c4079ae78ade353a9c5f3
I20260925 11:05:19.315311 11040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:19.344410 11040 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:19.345759 11040 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:19.351673 11040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145: Generated new TSK 0
I20260925 11:05:19.352459 11040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:05:19.406795 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:41769
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:19.725234 11055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:19.725504 11055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:19.725621 11055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:19.732762 11055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:19.732870 11055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:19.733016 11055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:05:19.742635 11055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:41769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:19.743889 11055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:19.745179 11055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:19.757510 11063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:19.757546 11061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:19.757567 11060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:19.758638 11055 server_base.cc:1034] running on GCE node
I20260925 11:05:19.759227 11055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:19.760129 11055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:19.761356 11055 hybrid_clock.cc:648] HybridClock initialized: now 1790334319761292 us; error 54 us; skew 500 ppm
I20260925 11:05:19.761768 11055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:19 dist-test-slave-trms krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:19.764753 11055 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:05:19.766353 11055 webserver.cc:460] Webserver started at http://127.24.159.193:40117/ using document root <none> and password file <none>
I20260925 11:05:19.766986 11055 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:19.767093 11055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:19.767349 11055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:19.770368 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "fd62ad792ba742ee914f212fa4293372"
format_stamp: "Formatted at 2026-09-25 11:05:19 on dist-test-slave-trms"
server_key: "37775bafc396fa38d34287a3854375fc"
server_key_iv: "13334aa0225f52437b51d915f7914cf8"
server_key_version: "encryptionkey@0"
I20260925 11:05:19.771272 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "fd62ad792ba742ee914f212fa4293372"
format_stamp: "Formatted at 2026-09-25 11:05:19 on dist-test-slave-trms"
server_key: "37775bafc396fa38d34287a3854375fc"
server_key_iv: "13334aa0225f52437b51d915f7914cf8"
server_key_version: "encryptionkey@0"
I20260925 11:05:19.776636 11055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260925 11:05:19.780072 11070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:19.781661 11055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:05:19.781804 11055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "fd62ad792ba742ee914f212fa4293372"
format_stamp: "Formatted at 2026-09-25 11:05:19 on dist-test-slave-trms"
server_key: "37775bafc396fa38d34287a3854375fc"
server_key_iv: "13334aa0225f52437b51d915f7914cf8"
server_key_version: "encryptionkey@0"
I20260925 11:05:19.781978 11055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:19.798966 11055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:19.802884 11055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:19.803125 11055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:19.803982 11055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:19.805243 11055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:19.805312 11055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:19.805402 11055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:19.805444 11055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:19.834540 11055 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:35477
I20260925 11:05:19.834599 11182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:35477 every 8 connection(s)
I20260925 11:05:19.835798 11055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:05:19.839727 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 11055
I20260925 11:05:19.839915 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:05:19.840444 25215 external_mini_cluster.cc:1428] Setting key 1d5d7185e9bcd012f968ad89af695fd6
Sep 25 11:05:19 dist-test-slave-trms krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:19.857213 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:19.838169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:45409 (local address 127.24.159.254:41769)
0925 11:05:19.838615 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:19.838620 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:19.839620 (+ 1000us) server_negotiation.cc:407] Connection header received
0925 11:05:19.841797 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:19.841803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:19.841876 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:19.841986 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:19.844004 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:19.844819 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:19.846897 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:19.847302 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:19.847398 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:19.851429 (+ 4031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:19.851436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:19.851444 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:19.851495 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:19.854246 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:19.854958 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:19.854962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:19.854964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:19.855085 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:19.855486 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:19.855489 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:19.855491 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:19.855763 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:19.855901 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:19.856617 (+ 716us) server_negotiation.cc:299] Negotiation successful
0925 11:05:19.856856 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":224,"threads_started":1}
I20260925 11:05:19.858242 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:19.838699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41769 (local address 127.24.159.193:45409)
0925 11:05:19.839401 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:19.839458 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:19.841540 (+ 2082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:19.842263 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:19.842275 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:19.843011 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:19.843804 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:19.843831 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:19.844989 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:19.844995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:19.846749 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:19.846763 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:19.847521 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:19.847528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:19.847791 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:19.848488 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:05:19.848521 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:19.851139 (+ 2618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:19.854386 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:19.854396 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:19.854410 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:19.854829 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:19.855187 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:19.855192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:19.855194 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:19.855372 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:19.855886 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:19.855894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:19.856253 (+ 359us) client_negotiation.cc:769] Sending connection context
0925 11:05:19.856504 (+ 251us) client_negotiation.cc:240] Negotiation successful
0925 11:05:19.857037 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":259,"threads_started":1}
I20260925 11:05:19.859833 11183 heartbeater.cc:347] Connected to a master server at 127.24.159.254:41769
I20260925 11:05:19.860116 11183 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:19.860822 11183 heartbeater.cc:511] Master 127.24.159.254:41769 requested a full tablet report, sending...
I20260925 11:05:19.863181 10994 ts_manager.cc:194] Registered new tserver with Master: fd62ad792ba742ee914f212fa4293372 (127.24.159.193:35477)
I20260925 11:05:19.865231 10994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:45409
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:05:19.894915 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:41769
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:20.212508 11189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:20.212790 11189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:20.212908 11189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:20.220477 11189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:20.220594 11189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:20.220741 11189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:05:20.230623 11189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:41769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:20.231914 11189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:20.233260 11189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:20.245831 11194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:20.245957 11197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:20.245951 11195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:20.246904 11189 server_base.cc:1034] running on GCE node
I20260925 11:05:20.247519 11189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:20.248409 11189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:20.249598 11189 hybrid_clock.cc:648] HybridClock initialized: now 1790334320249553 us; error 37 us; skew 500 ppm
I20260925 11:05:20.249905 11189 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:20 dist-test-slave-trms krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:20.252885 11189 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:05:20.254570 11189 webserver.cc:460] Webserver started at http://127.24.159.194:39379/ using document root <none> and password file <none>
I20260925 11:05:20.255422 11189 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:20.255566 11189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:20.255909 11189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:20.259644 11189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "264944b4d5cb4f8292a99c5f319a9bd9"
format_stamp: "Formatted at 2026-09-25 11:05:20 on dist-test-slave-trms"
server_key: "174c9fcdd79d6774c5f5d11537503adb"
server_key_iv: "c70bc829b1cd8560ffbfe7821633470c"
server_key_version: "encryptionkey@0"
I20260925 11:05:20.260568 11189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "264944b4d5cb4f8292a99c5f319a9bd9"
format_stamp: "Formatted at 2026-09-25 11:05:20 on dist-test-slave-trms"
server_key: "174c9fcdd79d6774c5f5d11537503adb"
server_key_iv: "c70bc829b1cd8560ffbfe7821633470c"
server_key_version: "encryptionkey@0"
I20260925 11:05:20.266742 11189 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260925 11:05:20.270979 11204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:20.272475 11189 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:05:20.272625 11189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "264944b4d5cb4f8292a99c5f319a9bd9"
format_stamp: "Formatted at 2026-09-25 11:05:20 on dist-test-slave-trms"
server_key: "174c9fcdd79d6774c5f5d11537503adb"
server_key_iv: "c70bc829b1cd8560ffbfe7821633470c"
server_key_version: "encryptionkey@0"
I20260925 11:05:20.272830 11189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:20.291975 11189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:20.295841 11189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:20.296108 11189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:20.297029 11189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:20.298357 11189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:20.298429 11189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:20.298518 11189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:20.298565 11189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:20.328722 11189 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:35837
I20260925 11:05:20.328763 11316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:35837 every 8 connection(s)
I20260925 11:05:20.330024 11189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:05:20.336189 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 11189
I20260925 11:05:20.336364 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:05:20.336766 25215 external_mini_cluster.cc:1428] Setting key 3d66b5e7fdb74d5eefdffb3f1d7a10f1
Sep 25 11:05:20 dist-test-slave-trms krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:20.349795 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.332567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:53993 (local address 127.24.159.254:41769)
0925 11:05:20.332763 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:20.332768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:20.333968 (+ 1200us) server_negotiation.cc:407] Connection header received
0925 11:05:20.335816 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:20.335822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:20.335919 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:20.336040 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:20.337661 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.338526 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.340074 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.340465 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.340573 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.343938 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:20.343943 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:20.343947 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:20.343981 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:20.346858 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:20.347555 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:20.347561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:20.347565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:20.347704 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:20.348107 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:20.348110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:20.348112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:20.348380 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:20.348525 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:20.349239 (+ 714us) server_negotiation.cc:299] Negotiation successful
0925 11:05:20.349467 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 11:05:20.350852 11319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.333077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41769 (local address 127.24.159.194:53993)
0925 11:05:20.333801 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:20.333848 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:20.335528 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:20.336233 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:20.336242 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:20.336810 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:20.337506 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.337533 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.338683 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.338688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.339931 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.339943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.340693 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.340700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.340918 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.341529 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:05:20.341556 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:20.343711 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:20.346994 (+ 3283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:20.347003 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:20.347021 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:20.347440 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:20.347809 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:20.347814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:20.347818 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:20.348002 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:20.348503 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:20.348511 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:20.348868 (+ 357us) client_negotiation.cc:769] Sending connection context
0925 11:05:20.349136 (+ 268us) client_negotiation.cc:240] Negotiation successful
0925 11:05:20.349674 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":291,"threads_started":1}
I20260925 11:05:20.352355 11317 heartbeater.cc:347] Connected to a master server at 127.24.159.254:41769
I20260925 11:05:20.352655 11317 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:20.353360 11317 heartbeater.cc:511] Master 127.24.159.254:41769 requested a full tablet report, sending...
I20260925 11:05:20.354861 10994 ts_manager.cc:194] Registered new tserver with Master: 264944b4d5cb4f8292a99c5f319a9bd9 (127.24.159.194:35837)
I20260925 11:05:20.355643 10994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:53993
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:05:20.389091 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:41769
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:35835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:20.705224 11322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:20.705514 11322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:20.705607 11322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:20.712788 11322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:20.712908 11322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:20.713019 11322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:05:20.722749 11322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:35835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:41769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:20.724012 11322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:20.725284 11322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:20.738205 11327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:20.738098 11328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:20.738253 11330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:20.738930 11322 server_base.cc:1034] running on GCE node
I20260925 11:05:20.739508 11322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:20.740432 11322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:20.741671 11322 hybrid_clock.cc:648] HybridClock initialized: now 1790334320741617 us; error 43 us; skew 500 ppm
I20260925 11:05:20.741971 11322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:20 dist-test-slave-trms krb5kdc[10949](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:20.744927 11322 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:05:20.746331 11322 webserver.cc:460] Webserver started at http://127.24.159.195:45443/ using document root <none> and password file <none>
I20260925 11:05:20.746984 11322 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:20.747104 11322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:20.747419 11322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:20.750252 11322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "ef9b21934bd945a08db8e2234ed8476d"
format_stamp: "Formatted at 2026-09-25 11:05:20 on dist-test-slave-trms"
server_key: "ad9a3a67ed641f8da7c52572e29bfedc"
server_key_iv: "1405fa0191c5525fc40e4e17d0d8ad84"
server_key_version: "encryptionkey@0"
I20260925 11:05:20.750978 11322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "ef9b21934bd945a08db8e2234ed8476d"
format_stamp: "Formatted at 2026-09-25 11:05:20 on dist-test-slave-trms"
server_key: "ad9a3a67ed641f8da7c52572e29bfedc"
server_key_iv: "1405fa0191c5525fc40e4e17d0d8ad84"
server_key_version: "encryptionkey@0"
I20260925 11:05:20.755864 11322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:05:20.759096 11337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:20.760349 11322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:05:20.760494 11322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "ef9b21934bd945a08db8e2234ed8476d"
format_stamp: "Formatted at 2026-09-25 11:05:20 on dist-test-slave-trms"
server_key: "ad9a3a67ed641f8da7c52572e29bfedc"
server_key_iv: "1405fa0191c5525fc40e4e17d0d8ad84"
server_key_version: "encryptionkey@0"
I20260925 11:05:20.760675 11322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:20.802843 11322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:20.806813 11322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:20.807086 11322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:20.808043 11322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:20.809378 11322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:20.809453 11322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:20.809567 11322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:20.809620 11322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:20.839299 11322 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:34153
I20260925 11:05:20.839344 11449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:34153 every 8 connection(s)
I20260925 11:05:20.840538 11322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:05:20.842110 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 11322
I20260925 11:05:20.842304 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:05:20.842715 25215 external_mini_cluster.cc:1428] Setting key 87b0104dc74e35a78def0f58c8b1d4f6
Sep 25 11:05:20 dist-test-slave-trms krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:20.859445 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.843045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:42209 (local address 127.24.159.254:41769)
0925 11:05:20.843247 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:20.843252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:20.844504 (+ 1252us) server_negotiation.cc:407] Connection header received
0925 11:05:20.846342 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:20.846346 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:20.846435 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:20.846513 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:20.847923 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.848704 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.850235 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.850640 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.850729 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.853864 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:20.853869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:20.853873 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:20.853905 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:20.856412 (+ 2507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:20.857077 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:20.857082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:20.857086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:20.857251 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:20.857702 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:20.857706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:20.857708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:20.858006 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:20.858142 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:20.858889 (+ 747us) server_negotiation.cc:299] Negotiation successful
0925 11:05:20.859114 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 11:05:20.860476 11452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.843528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:41769 (local address 127.24.159.195:42209)
0925 11:05:20.844348 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:20.844392 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:20.846070 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:20.846700 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:20.846708 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:20.847212 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:20.847752 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.847768 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.848872 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.848876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.850096 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.850108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.850813 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.850818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.851007 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.851606 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:05:20.851628 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:20.853663 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:20.856542 (+ 2879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:20.856549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:20.856562 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:20.856984 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:20.857357 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:20.857362 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:20.857365 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:20.857593 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:20.858141 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:20.858149 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:20.858536 (+ 387us) client_negotiation.cc:769] Sending connection context
0925 11:05:20.858782 (+ 246us) client_negotiation.cc:240] Negotiation successful
0925 11:05:20.859330 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":317,"threads_started":1}
I20260925 11:05:20.861788 11450 heartbeater.cc:347] Connected to a master server at 127.24.159.254:41769
I20260925 11:05:20.862063 11450 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:20.862773 11450 heartbeater.cc:511] Master 127.24.159.254:41769 requested a full tablet report, sending...
I20260925 11:05:20.864166 10994 ts_manager.cc:194] Registered new tserver with Master: ef9b21934bd945a08db8e2234ed8476d (127.24.159.195:34153)
I20260925 11:05:20.864959 10994 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:42209
I20260925 11:05:20.868237 11183 heartbeater.cc:503] Master 127.24.159.254:41769 was elected leader, sending a full tablet report...
I20260925 11:05:20.869987 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:05:20.885835 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.873830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.24.159.254:41769)
0925 11:05:20.873977 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:20.873981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:20.874472 (+ 491us) server_negotiation.cc:407] Connection header received
0925 11:05:20.874610 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:20.874614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:20.874676 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:20.874755 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:20.876019 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.876778 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.878101 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.878536 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.878627 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.881013 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:20.881018 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:20.881021 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:20.881052 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:20.883456 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:20.884158 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:20.884162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:20.884164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:20.884289 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:20.884608 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:20.884611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:20.884613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:20.884908 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:20.885091 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:20.885417 (+ 326us) server_negotiation.cc:299] Negotiation successful
0925 11:05:20.885573 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 11:05:20.889669 10993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:05:20.896448 10993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:05:20.918792 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.908037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60736 (local address 127.24.159.194:35837)
0925 11:05:20.908538 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:20.908547 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:20.909570 (+ 1023us) server_negotiation.cc:407] Connection header received
0925 11:05:20.909852 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:20.909861 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:20.910140 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:20.910346 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:20.911312 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.912844 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.915693 (+ 2849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.917165 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.917295 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.917475 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:20.918332 (+ 857us) server_negotiation.cc:299] Negotiation successful
0925 11:05:20.918499 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":233,"threads_started":1}
I20260925 11:05:20.918850 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.908076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:34153 (local address 127.0.0.1:47574)
0925 11:05:20.910145 (+ 2069us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:20.910161 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:20.910293 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:20.911199 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:20.911203 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:20.911228 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:20.911632 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.911644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.913430 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.913435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.915561 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.915571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.917661 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.917673 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.918064 (+ 391us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.918094 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 11:05:20.918229 (+ 135us) client_negotiation.cc:240] Negotiation successful
0925 11:05:20.918429 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1966,"mutex_wait_us":69,"thread_start_us":971,"threads_started":1}
I20260925 11:05:20.919534 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.908042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:35837 (local address 127.0.0.1:60736)
0925 11:05:20.909415 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:20.909463 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:20.909648 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:20.910607 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:20.910615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:20.910643 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:20.911108 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.911134 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.913011 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.913016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.915500 (+ 2484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.915519 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.917687 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.917693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.918022 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.918077 (+ 55us) client_negotiation.cc:769] Sending connection context
0925 11:05:20.919014 (+ 937us) client_negotiation.cc:240] Negotiation successful
0925 11:05:20.919158 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1124,"mutex_wait_us":104,"thread_start_us":2437,"threads_started":1}
I20260925 11:05:20.919548 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.908672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47574 (local address 127.24.159.195:34153)
0925 11:05:20.909405 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:20.909413 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:20.910401 (+ 988us) server_negotiation.cc:407] Connection header received
0925 11:05:20.910500 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:20.910507 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:20.910762 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:20.910944 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:20.911836 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.913283 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.915735 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.917155 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.917327 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.917475 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:20.919070 (+ 1595us) server_negotiation.cc:299] Negotiation successful
0925 11:05:20.919233 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":1410,"threads_started":1}
I20260925 11:05:20.923838 11252 tablet_service.cc:1467] Processing CreateTablet for tablet 5f5c7fa0741a42f6b6a6fabf14b88014 (DEFAULT_TABLE table=test-table [id=d1ff4ef7b9914eab86837540fea6510a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:20.924280 11385 tablet_service.cc:1467] Processing CreateTablet for tablet 5f5c7fa0741a42f6b6a6fabf14b88014 (DEFAULT_TABLE table=test-table [id=d1ff4ef7b9914eab86837540fea6510a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:20.925400 11252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5c7fa0741a42f6b6a6fabf14b88014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:20.925655 11385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5c7fa0741a42f6b6a6fabf14b88014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:20.927788 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.909934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:35477 (local address 127.0.0.1:42106)
0925 11:05:20.919865 (+ 9931us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:20.919883 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:20.919966 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:20.921175 (+ 1209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:20.921181 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:20.921203 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:20.921361 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.921372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.923281 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.923286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.925053 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:20.925068 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.927014 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.927024 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:20.927208 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.927241 (+ 33us) client_negotiation.cc:769] Sending connection context
0925 11:05:20.927362 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 11:05:20.927497 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9814,"thread_start_us":10711,"threads_started":1}
I20260925 11:05:20.927899 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:20.910299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.24.159.193:35477)
0925 11:05:20.920563 (+ 10264us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:20.920573 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:20.920600 (+ 27us) server_negotiation.cc:407] Connection header received
0925 11:05:20.920708 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:20.920716 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:20.920971 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:20.921525 (+ 554us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:20.921730 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.923134 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.925237 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:20.926688 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:20.927088 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:20.927265 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:20.927413 (+ 148us) server_negotiation.cc:299] Negotiation successful
0925 11:05:20.927578 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10101,"thread_start_us":9943,"threads_started":1}
I20260925 11:05:20.932248 11118 tablet_service.cc:1467] Processing CreateTablet for tablet 5f5c7fa0741a42f6b6a6fabf14b88014 (DEFAULT_TABLE table=test-table [id=d1ff4ef7b9914eab86837540fea6510a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:20.933820 11118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f5c7fa0741a42f6b6a6fabf14b88014. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:20.937413 11468 tablet_bootstrap.cc:492] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: Bootstrap starting.
I20260925 11:05:20.940864 11468 tablet_bootstrap.cc:654] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:20.941843 11468 log.cc:824] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:20.944545 11468 tablet_bootstrap.cc:492] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: No bootstrap required, opened a new log
I20260925 11:05:20.944855 11468 ts_tablet_manager.cc:1397] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260925 11:05:20.946352 11469 tablet_bootstrap.cc:492] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: Bootstrap starting.
I20260925 11:05:20.947685 11471 tablet_bootstrap.cc:492] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: Bootstrap starting.
I20260925 11:05:20.949836 11469 tablet_bootstrap.cc:654] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:20.950563 11471 tablet_bootstrap.cc:654] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:20.951088 11469 log.cc:824] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:20.951476 11471 log.cc:824] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:20.954751 11468 raft_consensus.cc:348] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } }
I20260925 11:05:20.955181 11468 raft_consensus.cc:374] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:20.955246 11471 tablet_bootstrap.cc:492] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: No bootstrap required, opened a new log
I20260925 11:05:20.955330 11468 raft_consensus.cc:729] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef9b21934bd945a08db8e2234ed8476d, State: Initialized, Role: FOLLOWER
I20260925 11:05:20.955515 11471 ts_tablet_manager.cc:1397] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260925 11:05:20.956012 11469 tablet_bootstrap.cc:492] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: No bootstrap required, opened a new log
I20260925 11:05:20.956034 11468 consensus_queue.cc:260] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } }
I20260925 11:05:20.956283 11469 ts_tablet_manager.cc:1397] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.006s
I20260925 11:05:20.957084 11450 heartbeater.cc:503] Master 127.24.159.254:41769 was elected leader, sending a full tablet report...
I20260925 11:05:20.957765 11468 ts_tablet_manager.cc:1428] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260925 11:05:20.964920 11471 raft_consensus.cc:348] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } }
I20260925 11:05:20.964987 11469 raft_consensus.cc:348] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } }
I20260925 11:05:20.965273 11471 raft_consensus.cc:374] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:20.965363 11471 raft_consensus.cc:729] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd62ad792ba742ee914f212fa4293372, State: Initialized, Role: FOLLOWER
I20260925 11:05:20.965404 11469 raft_consensus.cc:374] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:20.965500 11469 raft_consensus.cc:729] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264944b4d5cb4f8292a99c5f319a9bd9, State: Initialized, Role: FOLLOWER
I20260925 11:05:20.965927 11471 consensus_queue.cc:260] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } }
I20260925 11:05:20.965971 11469 consensus_queue.cc:260] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } }
I20260925 11:05:20.966930 11317 heartbeater.cc:503] Master 127.24.159.254:41769 was elected leader, sending a full tablet report...
I20260925 11:05:20.967595 11471 ts_tablet_manager.cc:1428] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: Time spent starting tablet: real 0.012s user 0.006s sys 0.003s
I20260925 11:05:20.967625 11469 ts_tablet_manager.cc:1428] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: Time spent starting tablet: real 0.011s user 0.003s sys 0.008s
I20260925 11:05:21.039054 11474 raft_consensus.cc:482] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:05:21.039361 11474 raft_consensus.cc:504] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } }
I20260925 11:05:21.041198 11474 leader_election.cc:288] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 264944b4d5cb4f8292a99c5f319a9bd9 (127.24.159.194:35837), fd62ad792ba742ee914f212fa4293372 (127.24.159.193:35477)
I20260925 11:05:21.048641 11452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:21.041557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:35837 (local address 127.24.159.195:56871)
0925 11:05:21.041850 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:21.041871 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:21.042026 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:21.042445 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:21.042451 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:21.042482 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:21.042644 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:21.042652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.044176 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.044183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:21.046381 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:21.046399 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.047898 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.047904 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:21.048108 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:21.048144 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 11:05:21.048225 (+ 81us) client_negotiation.cc:240] Negotiation successful
0925 11:05:21.048358 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153}
I20260925 11:05:21.048641 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:21.041639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:56871 (local address 127.24.159.194:35837)
0925 11:05:21.041841 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:21.041848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:21.041968 (+ 120us) server_negotiation.cc:407] Connection header received
0925 11:05:21.042174 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:21.042181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:21.042287 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:21.042387 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:21.042808 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.043982 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.046610 (+ 2628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.047572 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.047723 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:21.047802 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:21.048272 (+ 470us) server_negotiation.cc:299] Negotiation successful
0925 11:05:21.048400 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 11:05:21.049952 11272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5c7fa0741a42f6b6a6fabf14b88014" candidate_uuid: "ef9b21934bd945a08db8e2234ed8476d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" is_pre_election: true
I20260925 11:05:21.050628 11272 raft_consensus.cc:2454] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef9b21934bd945a08db8e2234ed8476d in term 0.
I20260925 11:05:21.051321 11341 leader_election.cc:302] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 264944b4d5cb4f8292a99c5f319a9bd9, ef9b21934bd945a08db8e2234ed8476d; no voters:
I20260925 11:05:21.051756 11474 raft_consensus.cc:2781] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:05:21.051927 11474 raft_consensus.cc:482] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:05:21.052038 11474 raft_consensus.cc:3037] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:21.052829 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:21.041974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:33703 (local address 127.24.159.193:35477)
0925 11:05:21.042317 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:21.042322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:21.046878 (+ 4556us) server_negotiation.cc:407] Connection header received
0925 11:05:21.047172 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:21.047178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:21.047261 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:21.047349 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:21.048068 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.049144 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.050860 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.051943 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.052099 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:21.052172 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:21.052459 (+ 287us) server_negotiation.cc:299] Negotiation successful
0925 11:05:21.052598 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260925 11:05:21.052935 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:21.041583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:35477 (local address 127.24.159.195:33703)
0925 11:05:21.046760 (+ 5177us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:21.046780 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:21.046996 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:21.047507 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:21.047512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:21.047540 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:21.047898 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:21.047908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.049284 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.049289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:21.050670 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:21.050683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.052138 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.052145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:21.052309 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:21.052336 (+ 27us) client_negotiation.cc:769] Sending connection context
0925 11:05:21.052418 (+ 82us) client_negotiation.cc:240] Negotiation successful
0925 11:05:21.052550 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5036,"thread_start_us":4790,"threads_started":1}
I20260925 11:05:21.053848 11138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5c7fa0741a42f6b6a6fabf14b88014" candidate_uuid: "ef9b21934bd945a08db8e2234ed8476d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd62ad792ba742ee914f212fa4293372" is_pre_election: true
I20260925 11:05:21.054369 11138 raft_consensus.cc:2454] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef9b21934bd945a08db8e2234ed8476d in term 0.
I20260925 11:05:21.054564 11474 raft_consensus.cc:504] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } }
I20260925 11:05:21.055193 11474 leader_election.cc:288] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [CANDIDATE]: Term 1 election: Requested vote from peers 264944b4d5cb4f8292a99c5f319a9bd9 (127.24.159.194:35837), fd62ad792ba742ee914f212fa4293372 (127.24.159.193:35477)
I20260925 11:05:21.055612 11138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5c7fa0741a42f6b6a6fabf14b88014" candidate_uuid: "ef9b21934bd945a08db8e2234ed8476d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd62ad792ba742ee914f212fa4293372"
I20260925 11:05:21.055649 11272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f5c7fa0741a42f6b6a6fabf14b88014" candidate_uuid: "ef9b21934bd945a08db8e2234ed8476d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264944b4d5cb4f8292a99c5f319a9bd9"
I20260925 11:05:21.055810 11138 raft_consensus.cc:3037] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:21.055855 11272 raft_consensus.cc:3037] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:21.058182 11138 raft_consensus.cc:2454] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef9b21934bd945a08db8e2234ed8476d in term 1.
I20260925 11:05:21.058418 11272 raft_consensus.cc:2454] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef9b21934bd945a08db8e2234ed8476d in term 1.
I20260925 11:05:21.058588 11338 leader_election.cc:302] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ef9b21934bd945a08db8e2234ed8476d, fd62ad792ba742ee914f212fa4293372; no voters:
I20260925 11:05:21.058898 11474 raft_consensus.cc:2781] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:21.059266 11474 raft_consensus.cc:686] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 1 LEADER]: Becoming Leader. State: Replica: ef9b21934bd945a08db8e2234ed8476d, State: Running, Role: LEADER
I20260925 11:05:21.059756 11474 consensus_queue.cc:237] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } }
I20260925 11:05:21.064136 10993 catalog_manager.cc:5495] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d reported cstate change: term changed from 0 to 1, leader changed from <none> to ef9b21934bd945a08db8e2234ed8476d (127.24.159.195). New cstate: current_term: 1 leader_uuid: "ef9b21934bd945a08db8e2234ed8476d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef9b21934bd945a08db8e2234ed8476d" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 34153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 } health_report { overall_health: UNKNOWN } } }
W20260925 11:05:21.081918 11318 tablet.cc:2369] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:05:21.086032 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:21.080488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47582 (local address 127.24.159.195:34153)
0925 11:05:21.080676 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:21.080682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:21.080758 (+ 76us) server_negotiation.cc:407] Connection header received
0925 11:05:21.080958 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:21.080962 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:21.081047 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:21.081171 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:05:21.081561 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.082385 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.083739 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.084140 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.084249 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:21.084853 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:05:21.085363 (+ 510us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:05:21.085517 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:21.085702 (+ 185us) server_negotiation.cc:299] Negotiation successful
0925 11:05:21.085750 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 11:05:21.087929 11184 tablet.cc:2369] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:05:21.092053 11451 tablet.cc:2369] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:05:21.093726 11138 raft_consensus.cc:1261] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [term 1 FOLLOWER]: Refusing update from remote peer ef9b21934bd945a08db8e2234ed8476d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:21.093729 11272 raft_consensus.cc:1261] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [term 1 FOLLOWER]: Refusing update from remote peer ef9b21934bd945a08db8e2234ed8476d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:21.094681 11474 consensus_queue.cc:1035] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd62ad792ba742ee914f212fa4293372" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 35477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:21.094965 11478 consensus_queue.cc:1035] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [LEADER]: Connected to new peer: Peer: permanent_uuid: "264944b4d5cb4f8292a99c5f319a9bd9" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 35837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:21.122130 10994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:21.122421 10994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:21.125696 10994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0a7e3a94b312477ea46396345b320145: Sending DeleteTablet for 3 replicas of tablet 5f5c7fa0741a42f6b6a6fabf14b88014
I20260925 11:05:21.126766 11385 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f5c7fa0741a42f6b6a6fabf14b88014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:21 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:47574
I20260925 11:05:21.126787 11252 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f5c7fa0741a42f6b6a6fabf14b88014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:21 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:60736
I20260925 11:05:21.127486 11118 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f5c7fa0741a42f6b6a6fabf14b88014 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:21 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:42106
I20260925 11:05:21.130891 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 11055
I20260925 11:05:21.139436 11493 tablet_replica.cc:330] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: stopping tablet replica
I20260925 11:05:21.140570 11493 raft_consensus.cc:2229] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:21.141835 11493 raft_consensus.cc:2258] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:21.142666 11492 tablet_replica.cc:330] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: stopping tablet replica
I20260925 11:05:21.143124 11492 raft_consensus.cc:2229] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:21.143457 11491 tablet_replica.cc:330] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: stopping tablet replica
I20260925 11:05:21.143708 11492 raft_consensus.cc:2258] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:21.143862 11491 raft_consensus.cc:2229] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:05:21.144536 11491 raft_consensus.cc:2258] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:21.144811 11493 ts_tablet_manager.cc:1905] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:21.145951 11492 ts_tablet_manager.cc:1905] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:21.146603 11491 ts_tablet_manager.cc:1905] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:21.151856 11493 ts_tablet_manager.cc:1918] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:21.152097 11493 log.cc:1196] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/5f5c7fa0741a42f6b6a6fabf14b88014
I20260925 11:05:21.152671 11493 ts_tablet_manager.cc:1939] T 5f5c7fa0741a42f6b6a6fabf14b88014 P fd62ad792ba742ee914f212fa4293372: Deleting consensus metadata
I20260925 11:05:21.152930 11492 ts_tablet_manager.cc:1918] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:21.153154 11492 log.cc:1196] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/5f5c7fa0741a42f6b6a6fabf14b88014
I20260925 11:05:21.153590 11491 ts_tablet_manager.cc:1918] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:21.153795 11491 log.cc:1196] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/5f5c7fa0741a42f6b6a6fabf14b88014
I20260925 11:05:21.154261 11492 ts_tablet_manager.cc:1939] T 5f5c7fa0741a42f6b6a6fabf14b88014 P 264944b4d5cb4f8292a99c5f319a9bd9: Deleting consensus metadata
I20260925 11:05:21.154341 11491 ts_tablet_manager.cc:1939] T 5f5c7fa0741a42f6b6a6fabf14b88014 P ef9b21934bd945a08db8e2234ed8476d: Deleting consensus metadata
I20260925 11:05:21.155089 10978 catalog_manager.cc:4841] TS fd62ad792ba742ee914f212fa4293372 (127.24.159.193:35477): tablet 5f5c7fa0741a42f6b6a6fabf14b88014 (table test-table [id=d1ff4ef7b9914eab86837540fea6510a]) successfully deleted
I20260925 11:05:21.155985 10978 catalog_manager.cc:4841] TS ef9b21934bd945a08db8e2234ed8476d (127.24.159.195:34153): tablet 5f5c7fa0741a42f6b6a6fabf14b88014 (table test-table [id=d1ff4ef7b9914eab86837540fea6510a]) successfully deleted
I20260925 11:05:21.156097 10981 catalog_manager.cc:4841] TS 264944b4d5cb4f8292a99c5f319a9bd9 (127.24.159.194:35837): tablet 5f5c7fa0741a42f6b6a6fabf14b88014 (table test-table [id=d1ff4ef7b9914eab86837540fea6510a]) successfully deleted
Sep 25 11:05:21 dist-test-slave-trms krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:05:21.168223 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:21.131890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42110 (local address 127.24.159.193:35477)
0925 11:05:21.132371 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:21.132377 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:21.142438 (+ 10061us) server_negotiation.cc:407] Connection header received
0925 11:05:21.142746 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:21.142752 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:21.142837 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:21.142916 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:21.156155 (+ 13239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.156918 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.158287 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.158704 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.158812 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:21.162699 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:21.162711 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:21.162725 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:21.162756 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:21.165555 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:21.166183 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:21.166249 (+ 66us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:21.166253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:21.166383 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:21.166735 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:21.166740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:21.166742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:21.167084 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:21.167362 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:21.167668 (+ 306us) server_negotiation.cc:299] Negotiation successful
0925 11:05:21.167865 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341}
I20260925 11:05:21.169107 11178 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:21.257017 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 11055
I20260925 11:05:21.271421 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 11189
Sep 25 11:05:21 dist-test-slave-trms krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:05:21.287102 11462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:21.272211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60752 (local address 127.24.159.194:35837)
0925 11:05:21.272385 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:21.272391 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:21.272456 (+ 65us) server_negotiation.cc:407] Connection header received
0925 11:05:21.272650 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:21.272656 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:21.272736 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:21.272821 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:21.274079 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.275017 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.276292 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.276800 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.276914 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:21.281606 (+ 4692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:21.281630 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:21.281647 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:21.281683 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:21.284419 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:21.285072 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:21.285080 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:21.285082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:21.285214 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:21.285565 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:21.285569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:21.285571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:21.285947 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:21.286250 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:21.286554 (+ 304us) server_negotiation.cc:299] Negotiation successful
0925 11:05:21.286786 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260925 11:05:21.287812 11312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:21.375660 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 11189
I20260925 11:05:21.396303 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 11322
Sep 25 11:05:21 dist-test-slave-trms krb5kdc[10949](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:05:21.413437 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:21.397411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47592 (local address 127.24.159.195:34153)
0925 11:05:21.397670 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:21.397676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:21.397710 (+ 34us) server_negotiation.cc:407] Connection header received
0925 11:05:21.397772 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:21.397777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:21.397864 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:21.397956 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:21.399482 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.400405 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.401814 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:21.402347 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:21.402478 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:21.407282 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:21.407295 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:21.407311 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:21.407349 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:21.410364 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:21.411255 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:21.411263 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:21.411266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:21.411455 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:21.411899 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:21.411903 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:21.411906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:21.412296 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:21.412558 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:21.412931 (+ 373us) server_negotiation.cc:299] Negotiation successful
0925 11:05:21.413112 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260925 11:05:21.414106 11445 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:21.534049 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 11322
I20260925 11:05:21.550381 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 10962
I20260925 11:05:21.551496 11024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:21.657827 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 10962
2026-09-25T11:05:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4076 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-25T11:05:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:21Z Disabled control of system clock
I20260925 11:05:21.726903 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:46665
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:46665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 11:05:22.045758 11502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:22.046036 11502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:22.046180 11502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:22.053435 11502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:22.053536 11502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:22.053597 11502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:22.053673 11502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:22.053725 11502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:22.062763 11502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:46665
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:46665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:22.063948 11502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:22.065640 11502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:22.076601 11508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:22.076622 11507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:22.077070 11510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:22.077580 11502 server_base.cc:1034] running on GCE node
I20260925 11:05:22.078364 11502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:22.079710 11502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:22.080906 11502 hybrid_clock.cc:648] HybridClock initialized: now 1790334322080850 us; error 49 us; skew 500 ppm
I20260925 11:05:22.081228 11502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:22.083410 11502 webserver.cc:460] Webserver started at http://127.24.159.254:45711/ using document root <none> and password file <none>
I20260925 11:05:22.084043 11502 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:22.084133 11502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:22.084390 11502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:22.087255 11502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "e32e238fafce4a6292937808b83db035"
format_stamp: "Formatted at 2026-09-25 11:05:22 on dist-test-slave-trms"
server_key: "a71e8f8bfd3e84f0d571df7012b91eae"
server_key_iv: "d00f1ae3f7482bb48770fafdbb212c4a"
server_key_version: "encryptionkey@0"
I20260925 11:05:22.088035 11502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "e32e238fafce4a6292937808b83db035"
format_stamp: "Formatted at 2026-09-25 11:05:22 on dist-test-slave-trms"
server_key: "a71e8f8bfd3e84f0d571df7012b91eae"
server_key_iv: "d00f1ae3f7482bb48770fafdbb212c4a"
server_key_version: "encryptionkey@0"
I20260925 11:05:22.093113 11502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:05:22.096546 11516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:22.097980 11502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260925 11:05:22.098166 11502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "e32e238fafce4a6292937808b83db035"
format_stamp: "Formatted at 2026-09-25 11:05:22 on dist-test-slave-trms"
server_key: "a71e8f8bfd3e84f0d571df7012b91eae"
server_key_iv: "d00f1ae3f7482bb48770fafdbb212c4a"
server_key_version: "encryptionkey@0"
I20260925 11:05:22.098431 11502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:22.113277 11502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:22.114123 11502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:22.114454 11502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:22.130872 11502 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:46665
I20260925 11:05:22.130894 11567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:46665 every 8 connection(s)
I20260925 11:05:22.132158 11502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:22.135735 11568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:22.138612 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 11502
I20260925 11:05:22.138787 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:22.139362 25215 external_mini_cluster.cc:1428] Setting key 8d34a5a1d714aedaff5bf55a38933484
I20260925 11:05:22.147331 11568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035: Bootstrap starting.
I20260925 11:05:22.150385 11568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:22.151461 11568 log.cc:824] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:22.152562 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:22.141342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55362 (local address 127.24.159.254:46665)
0925 11:05:22.142344 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:22.142355 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:22.142399 (+ 44us) server_negotiation.cc:407] Connection header received
0925 11:05:22.144060 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:22.144087 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:22.144586 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:22.144955 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:22.145506 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:22.146793 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:22.148134 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:22.148823 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:22.148983 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:22.150081 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:22.150089 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:22.150102 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:22.150709 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:22.150804 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:22.150824 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:22.150920 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:22.151151 (+ 231us) server_negotiation.cc:299] Negotiation successful
0925 11:05:22.151442 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":356,"threads_started":1}
I20260925 11:05:22.154569 11568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035: No bootstrap required, opened a new log
I20260925 11:05:22.161605 11568 raft_consensus.cc:348] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32e238fafce4a6292937808b83db035" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46665 } }
I20260925 11:05:22.161927 11568 raft_consensus.cc:374] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:22.162004 11568 raft_consensus.cc:729] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32e238fafce4a6292937808b83db035, State: Initialized, Role: FOLLOWER
I20260925 11:05:22.162497 11568 consensus_queue.cc:260] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32e238fafce4a6292937808b83db035" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46665 } }
I20260925 11:05:22.162699 11568 raft_consensus.cc:388] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:22.162791 11568 raft_consensus.cc:482] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:22.162907 11568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:22.164925 11568 raft_consensus.cc:504] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32e238fafce4a6292937808b83db035" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46665 } }
I20260925 11:05:22.165460 11568 leader_election.cc:302] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e32e238fafce4a6292937808b83db035; no voters:
I20260925 11:05:22.165993 11568 leader_election.cc:288] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:22.166110 11573 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:22.166780 11573 raft_consensus.cc:686] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [term 1 LEADER]: Becoming Leader. State: Replica: e32e238fafce4a6292937808b83db035, State: Running, Role: LEADER
I20260925 11:05:22.167263 11573 consensus_queue.cc:237] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32e238fafce4a6292937808b83db035" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46665 } }
I20260925 11:05:22.167846 11568 sys_catalog.cc:564] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:22.169981 11574 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e32e238fafce4a6292937808b83db035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32e238fafce4a6292937808b83db035" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46665 } } }
I20260925 11:05:22.170368 11574 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:22.171448 11575 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e32e238fafce4a6292937808b83db035. Latest consensus state: current_term: 1 leader_uuid: "e32e238fafce4a6292937808b83db035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32e238fafce4a6292937808b83db035" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 46665 } } }
I20260925 11:05:22.171744 11575 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:22.175292 11581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:22.179572 11581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:22.187124 11581 catalog_manager.cc:1347] Generated new cluster ID: f59f9c874d3d428ba6d6c1a64e2b1370
I20260925 11:05:22.187227 11581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:22.207581 11581 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:22.208921 11581 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:22.218562 11581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035: Generated new TSK 0
I20260925 11:05:22.219419 11581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:22.237623 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:46665
--builtin_ntp_servers=127.24.159.212:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 11:05:22.558305 11592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:22.558588 11592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:22.558717 11592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:22.565804 11592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:22.565907 11592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:22.566042 11592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:05:22.575191 11592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:46665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:22.576504 11592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:22.578300 11592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:22.591133 11597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:22.591074 11600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:22.591100 11598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:22.592072 11592 server_base.cc:1034] running on GCE node
I20260925 11:05:22.592689 11592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:22.593608 11592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:22.594805 11592 hybrid_clock.cc:648] HybridClock initialized: now 1790334322594753 us; error 39 us; skew 500 ppm
I20260925 11:05:22.595104 11592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:22.597242 11592 webserver.cc:460] Webserver started at http://127.24.159.193:41459/ using document root <none> and password file <none>
I20260925 11:05:22.597893 11592 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:22.597990 11592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:22.598335 11592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:22.601116 11592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "2bc232fa0317409c9eca9e3d9750eec1"
format_stamp: "Formatted at 2026-09-25 11:05:22 on dist-test-slave-trms"
server_key: "86cdca2ad5ad9f3bebb1201c4b9a6b7d"
server_key_iv: "06cbfdedf70bdea456a18adf54920958"
server_key_version: "encryptionkey@0"
I20260925 11:05:22.601944 11592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "2bc232fa0317409c9eca9e3d9750eec1"
format_stamp: "Formatted at 2026-09-25 11:05:22 on dist-test-slave-trms"
server_key: "86cdca2ad5ad9f3bebb1201c4b9a6b7d"
server_key_iv: "06cbfdedf70bdea456a18adf54920958"
server_key_version: "encryptionkey@0"
I20260925 11:05:22.606906 11592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260925 11:05:22.610106 11606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:22.611400 11592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:05:22.611536 11592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "2bc232fa0317409c9eca9e3d9750eec1"
format_stamp: "Formatted at 2026-09-25 11:05:22 on dist-test-slave-trms"
server_key: "86cdca2ad5ad9f3bebb1201c4b9a6b7d"
server_key_iv: "06cbfdedf70bdea456a18adf54920958"
server_key_version: "encryptionkey@0"
I20260925 11:05:22.611721 11592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:22.624413 11592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:22.625182 11592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:22.625447 11592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:22.626324 11592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:22.627666 11592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:22.627758 11592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:22.627866 11592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:22.627934 11592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:22.657017 11592 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:43521
I20260925 11:05:22.657056 11718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:43521 every 8 connection(s)
I20260925 11:05:22.658341 11592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:05:22.663455 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 11592
I20260925 11:05:22.663625 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:05:22.664147 25215 external_mini_cluster.cc:1428] Setting key ace7e000ff87b511c19b0a3661b04157
I20260925 11:05:22.666996 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:46665
--builtin_ntp_servers=127.24.159.212:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 11:05:22.675771 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:22.660704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:53021 (local address 127.24.159.254:46665)
0925 11:05:22.661133 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:22.661138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:22.662230 (+ 1092us) server_negotiation.cc:407] Connection header received
0925 11:05:22.664428 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:22.664433 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:22.664508 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:22.664605 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:22.665813 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:22.666673 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:22.668957 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:22.669453 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:22.669563 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:22.674897 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:22.674904 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:22.674909 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:22.674934 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:22.674987 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:22.675003 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:22.675100 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:22.675303 (+ 203us) server_negotiation.cc:299] Negotiation successful
0925 11:05:22.675417 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":220,"threads_started":1}
I20260925 11:05:22.676756 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:22.661211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:46665 (local address 127.24.159.193:53021)
0925 11:05:22.662064 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:22.662105 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:22.664095 (+ 1990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:22.664862 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:22.664871 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:22.664996 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:22.665651 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:22.665670 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:22.666842 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:22.666848 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:22.668775 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:22.668793 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:22.669677 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:22.669684 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:22.669968 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:22.674534 (+ 4566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:05:22.674536 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:22.674580 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:05:22.674692 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:22.675106 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:22.675115 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:22.675121 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 11:05:22.675269 (+ 148us) client_negotiation.cc:240] Negotiation successful
0925 11:05:22.675633 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":245,"threads_started":1}
I20260925 11:05:22.678423 11719 heartbeater.cc:347] Connected to a master server at 127.24.159.254:46665
I20260925 11:05:22.678712 11719 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:22.679471 11719 heartbeater.cc:511] Master 127.24.159.254:46665 requested a full tablet report, sending...
I20260925 11:05:22.681739 11533 ts_manager.cc:194] Registered new tserver with Master: 2bc232fa0317409c9eca9e3d9750eec1 (127.24.159.193:43521)
I20260925 11:05:22.683724 11533 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.193:53021
W20260925 11:05:22.986591 11723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:22.986860 11723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:22.986951 11723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:22.994082 11723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:22.994206 11723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:22.994341 11723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:05:23.003487 11723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:46665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:23.004700 11723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:23.006480 11723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:23.018988 11728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:23.019035 11729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:23.019013 11731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:23.019682 11723 server_base.cc:1034] running on GCE node
I20260925 11:05:23.020290 11723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:23.021298 11723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:23.022496 11723 hybrid_clock.cc:648] HybridClock initialized: now 1790334323022451 us; error 33 us; skew 500 ppm
I20260925 11:05:23.022782 11723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:23.024951 11723 webserver.cc:460] Webserver started at http://127.24.159.194:39813/ using document root <none> and password file <none>
I20260925 11:05:23.025591 11723 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:23.025676 11723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:23.025938 11723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:23.028788 11723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "86b18d1bb394426493fca2cdbbdc0ff8"
format_stamp: "Formatted at 2026-09-25 11:05:23 on dist-test-slave-trms"
server_key: "d89a2d91c2912129800655d4590b8773"
server_key_iv: "092fa3734eff7bf55f95d8313f0fe994"
server_key_version: "encryptionkey@0"
I20260925 11:05:23.029547 11723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "86b18d1bb394426493fca2cdbbdc0ff8"
format_stamp: "Formatted at 2026-09-25 11:05:23 on dist-test-slave-trms"
server_key: "d89a2d91c2912129800655d4590b8773"
server_key_iv: "092fa3734eff7bf55f95d8313f0fe994"
server_key_version: "encryptionkey@0"
I20260925 11:05:23.034695 11723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 11:05:23.038072 11737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:23.039363 11723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 11:05:23.039511 11723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "86b18d1bb394426493fca2cdbbdc0ff8"
format_stamp: "Formatted at 2026-09-25 11:05:23 on dist-test-slave-trms"
server_key: "d89a2d91c2912129800655d4590b8773"
server_key_iv: "092fa3734eff7bf55f95d8313f0fe994"
server_key_version: "encryptionkey@0"
I20260925 11:05:23.039705 11723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:23.061545 11723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:23.062424 11723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:23.062703 11723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:23.063624 11723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:23.065049 11723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:23.065135 11723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:23.065237 11723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:23.065312 11723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:23.095007 11723 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:41465
I20260925 11:05:23.095082 11849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:41465 every 8 connection(s)
I20260925 11:05:23.096285 11723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:05:23.102360 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 11723
I20260925 11:05:23.102514 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:05:23.102983 25215 external_mini_cluster.cc:1428] Setting key f2b007bbe8bb0b03aa2c7ffe7321ad59
I20260925 11:05:23.105906 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:46665
--builtin_ntp_servers=127.24.159.212:36163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 11:05:23.109262 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.098693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:55765 (local address 127.24.159.254:46665)
0925 11:05:23.098851 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.098856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.100256 (+ 1400us) server_negotiation.cc:407] Connection header received
0925 11:05:23.102003 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.102009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.102098 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.102215 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:23.103404 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.104376 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.106620 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.107014 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.107112 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.108583 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:23.108588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:23.108591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:23.108608 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:23.108641 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:23.108653 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:23.108718 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.108929 (+ 211us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.109005 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260925 11:05:23.110883 11852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.099207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:46665 (local address 127.24.159.194:55765)
0925 11:05:23.100106 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.100143 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:23.101723 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:23.102454 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:23.102464 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:23.102581 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:23.103243 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.103263 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.104579 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.104585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.106479 (+ 1894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.106494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.107233 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.107240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.107528 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.108246 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:05:23.108249 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:23.108302 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:05:23.108407 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:23.108736 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:23.108745 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:23.108752 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 11:05:23.108923 (+ 171us) client_negotiation.cc:240] Negotiation successful
0925 11:05:23.109321 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":256,"threads_started":1}
I20260925 11:05:23.112298 11850 heartbeater.cc:347] Connected to a master server at 127.24.159.254:46665
I20260925 11:05:23.112629 11850 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:23.113494 11850 heartbeater.cc:511] Master 127.24.159.254:46665 requested a full tablet report, sending...
I20260925 11:05:23.115027 11533 ts_manager.cc:194] Registered new tserver with Master: 86b18d1bb394426493fca2cdbbdc0ff8 (127.24.159.194:41465)
I20260925 11:05:23.116080 11533 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.194:55765
W20260925 11:05:23.428658 11853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:23.428988 11853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:23.429142 11853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:23.436432 11853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:23.436527 11853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:23.436661 11853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:05:23.445735 11853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:46665
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:23.446985 11853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:23.448697 11853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:23.460973 11858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:23.461047 11859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:23.461076 11861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:23.461935 11853 server_base.cc:1034] running on GCE node
I20260925 11:05:23.462524 11853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:23.463393 11853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:23.464632 11853 hybrid_clock.cc:648] HybridClock initialized: now 1790334323464576 us; error 41 us; skew 500 ppm
I20260925 11:05:23.464952 11853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:23.467242 11853 webserver.cc:460] Webserver started at http://127.24.159.195:42371/ using document root <none> and password file <none>
I20260925 11:05:23.467928 11853 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:23.468034 11853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:23.468335 11853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:23.471273 11853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "c80c0bbaff0447e59f1a7b400194739a"
format_stamp: "Formatted at 2026-09-25 11:05:23 on dist-test-slave-trms"
server_key: "9cb1d735c2bd5ed86defae25062a7565"
server_key_iv: "83ce4b963933fd32dc49be575cf19c67"
server_key_version: "encryptionkey@0"
I20260925 11:05:23.472029 11853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "c80c0bbaff0447e59f1a7b400194739a"
format_stamp: "Formatted at 2026-09-25 11:05:23 on dist-test-slave-trms"
server_key: "9cb1d735c2bd5ed86defae25062a7565"
server_key_iv: "83ce4b963933fd32dc49be575cf19c67"
server_key_version: "encryptionkey@0"
I20260925 11:05:23.477097 11853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 11:05:23.480533 11867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:23.481890 11853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:05:23.482044 11853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "c80c0bbaff0447e59f1a7b400194739a"
format_stamp: "Formatted at 2026-09-25 11:05:23 on dist-test-slave-trms"
server_key: "9cb1d735c2bd5ed86defae25062a7565"
server_key_iv: "83ce4b963933fd32dc49be575cf19c67"
server_key_version: "encryptionkey@0"
I20260925 11:05:23.482299 11853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:23.500787 11853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:23.501632 11853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:23.501914 11853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:23.502902 11853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:23.504371 11853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:23.504477 11853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:23.504585 11853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:23.504652 11853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:23.534363 11853 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:36813
I20260925 11:05:23.534415 11979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:36813 every 8 connection(s)
I20260925 11:05:23.535651 11853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:05:23.540829 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 11853
I20260925 11:05:23.540990 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:05:23.541417 25215 external_mini_cluster.cc:1428] Setting key b69bfd1fe89774f247c5840f2c005f4f
I20260925 11:05:23.547670 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.538120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:53615 (local address 127.24.159.254:46665)
0925 11:05:23.538330 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.538334 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.539750 (+ 1416us) server_negotiation.cc:407] Connection header received
0925 11:05:23.541544 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.541550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.541637 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.541723 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:23.543019 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.543824 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.545389 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.545783 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.545879 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.547063 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:23.547067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:23.547071 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:23.547087 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:23.547116 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:23.547127 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:23.547193 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.547354 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.547432 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 11:05:23.548568 11982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.538698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:46665 (local address 127.24.159.195:53615)
0925 11:05:23.539590 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.539633 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:23.541282 (+ 1649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:23.541969 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:23.541978 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:23.542067 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:23.542844 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.542863 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.543992 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.543997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.545274 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.545285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.545945 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.545950 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.546156 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.546815 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:05:23.546818 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:23.546854 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:05:23.546931 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:23.547198 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:23.547204 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:23.547209 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 11:05:23.547330 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 11:05:23.547601 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":250,"threads_started":1}
I20260925 11:05:23.549752 11980 heartbeater.cc:347] Connected to a master server at 127.24.159.254:46665
I20260925 11:05:23.550028 11980 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:23.550763 11980 heartbeater.cc:511] Master 127.24.159.254:46665 requested a full tablet report, sending...
I20260925 11:05:23.552088 11533 ts_manager.cc:194] Registered new tserver with Master: c80c0bbaff0447e59f1a7b400194739a (127.24.159.195:36813)
I20260925 11:05:23.552862 11533 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.195:53615
I20260925 11:05:23.556110 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:05:23.565380 11721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.559779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55366 (local address 127.24.159.254:46665)
0925 11:05:23.559958 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.559964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.560367 (+ 403us) server_negotiation.cc:407] Connection header received
0925 11:05:23.560518 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.560522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.560590 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.560667 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:23.561143 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.561818 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.563168 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.563555 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.563659 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.564711 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:23.564716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:23.564719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:23.564737 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:23.564767 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:23.564778 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:23.564903 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.565072 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.565126 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 11:05:23.569002 11533 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:05:23.575774 11533 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:05:23.597877 11993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.587744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49268 (local address 127.24.159.193:43521)
0925 11:05:23.588322 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.588331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.588469 (+ 138us) server_negotiation.cc:407] Connection header received
0925 11:05:23.588850 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.588856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.589041 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.589223 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:23.589998 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.591654 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.594675 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.596318 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.596456 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.596691 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.597366 (+ 675us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.597541 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":326,"threads_started":1}
I20260925 11:05:23.597891 11992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.587598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43521 (local address 127.0.0.1:49268)
0925 11:05:23.588335 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.588366 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:23.588548 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:23.589245 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:23.589252 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:23.589281 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:23.589801 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.589815 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.591860 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.591867 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.594444 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.594463 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.596854 (+ 2391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.596860 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.597097 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.597153 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 11:05:23.597289 (+ 136us) client_negotiation.cc:240] Negotiation successful
0925 11:05:23.597512 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":763,"threads_started":1}
I20260925 11:05:23.601788 11654 tablet_service.cc:1467] Processing CreateTablet for tablet a777fbd40bc145b9b1cfc955970f1337 (DEFAULT_TABLE table=test-table [id=90b0a40d84e34800a92d031a18dcee87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:23.603170 11654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a777fbd40bc145b9b1cfc955970f1337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:23.607334 11996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.589939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:36813 (local address 127.0.0.1:40716)
0925 11:05:23.596640 (+ 6701us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.596659 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:23.596767 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:23.600343 (+ 3576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:23.600347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:23.600370 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:23.600670 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.600679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.602059 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.602065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.604415 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.604434 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.606439 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.606446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.606641 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.606702 (+ 61us) client_negotiation.cc:769] Sending connection context
0925 11:05:23.606792 (+ 90us) client_negotiation.cc:240] Negotiation successful
0925 11:05:23.606928 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6586,"mutex_wait_us":24,"thread_start_us":6458,"threads_started":1}
I20260925 11:05:23.607537 11995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.589476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.24.159.194:41465)
0925 11:05:23.596964 (+ 7488us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.596971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.596998 (+ 27us) server_negotiation.cc:407] Connection header received
0925 11:05:23.597075 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.597080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.597278 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.598123 (+ 845us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:23.598434 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.602174 (+ 3740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.604656 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.606069 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.606785 (+ 716us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.606977 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.607043 (+ 66us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.607200 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7343,"thread_start_us":7224,"threads_started":1}
I20260925 11:05:23.608285 11997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.590419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40716 (local address 127.24.159.195:36813)
0925 11:05:23.599891 (+ 9472us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.599897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.599926 (+ 29us) server_negotiation.cc:407] Connection header received
0925 11:05:23.599996 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.600001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.600170 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.600338 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:23.600831 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.601934 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.604664 (+ 2730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.606090 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.606311 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.606533 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.607799 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.607966 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9318,"thread_start_us":9178,"threads_started":1}
I20260925 11:05:23.607374 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.589105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:41465 (local address 127.0.0.1:33076)
0925 11:05:23.589868 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.589884 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:23.589976 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:23.597490 (+ 7514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:23.597495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:23.597521 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:23.597899 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.597911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.602322 (+ 4411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.602328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.604439 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.604448 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.606518 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.606531 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.606737 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.606773 (+ 36us) client_negotiation.cc:769] Sending connection context
0925 11:05:23.606860 (+ 87us) client_negotiation.cc:240] Negotiation successful
0925 11:05:23.606990 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":78,"thread_start_us":462,"threads_started":1}
I20260925 11:05:23.612231 11915 tablet_service.cc:1467] Processing CreateTablet for tablet a777fbd40bc145b9b1cfc955970f1337 (DEFAULT_TABLE table=test-table [id=90b0a40d84e34800a92d031a18dcee87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:23.612231 11785 tablet_service.cc:1467] Processing CreateTablet for tablet a777fbd40bc145b9b1cfc955970f1337 (DEFAULT_TABLE table=test-table [id=90b0a40d84e34800a92d031a18dcee87]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:23.613633 11915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a777fbd40bc145b9b1cfc955970f1337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:23.613633 11785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a777fbd40bc145b9b1cfc955970f1337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:23.615775 11998 tablet_bootstrap.cc:492] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1: Bootstrap starting.
I20260925 11:05:23.619314 11998 tablet_bootstrap.cc:654] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:23.620261 11998 log.cc:824] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:23.622706 11998 tablet_bootstrap.cc:492] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1: No bootstrap required, opened a new log
I20260925 11:05:23.622953 11998 ts_tablet_manager.cc:1397] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260925 11:05:23.627895 12000 tablet_bootstrap.cc:492] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: Bootstrap starting.
I20260925 11:05:23.631443 12000 tablet_bootstrap.cc:654] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:23.632202 11998 raft_consensus.cc:348] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } }
I20260925 11:05:23.632651 11998 raft_consensus.cc:374] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:23.632676 12000 log.cc:824] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:23.632759 11998 raft_consensus.cc:729] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bc232fa0317409c9eca9e3d9750eec1, State: Initialized, Role: FOLLOWER
I20260925 11:05:23.633386 11998 consensus_queue.cc:260] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } }
I20260925 11:05:23.634048 12001 tablet_bootstrap.cc:492] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: Bootstrap starting.
I20260925 11:05:23.635671 12000 tablet_bootstrap.cc:492] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: No bootstrap required, opened a new log
I20260925 11:05:23.635912 12000 ts_tablet_manager.cc:1397] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 11:05:23.638474 12001 tablet_bootstrap.cc:654] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:23.639758 12001 log.cc:824] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:23.641292 11719 heartbeater.cc:503] Master 127.24.159.254:46665 was elected leader, sending a full tablet report...
I20260925 11:05:23.642692 11998 ts_tablet_manager.cc:1428] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1: Time spent starting tablet: real 0.020s user 0.006s sys 0.012s
I20260925 11:05:23.642848 12001 tablet_bootstrap.cc:492] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: No bootstrap required, opened a new log
I20260925 11:05:23.643181 12001 ts_tablet_manager.cc:1397] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260925 11:05:23.643682 12000 raft_consensus.cc:348] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } }
I20260925 11:05:23.644021 12000 raft_consensus.cc:374] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:23.644127 12000 raft_consensus.cc:729] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b18d1bb394426493fca2cdbbdc0ff8, State: Initialized, Role: FOLLOWER
I20260925 11:05:23.644783 12000 consensus_queue.cc:260] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } }
I20260925 11:05:23.645794 11850 heartbeater.cc:503] Master 127.24.159.254:46665 was elected leader, sending a full tablet report...
I20260925 11:05:23.646430 12000 ts_tablet_manager.cc:1428] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: Time spent starting tablet: real 0.010s user 0.007s sys 0.005s
I20260925 11:05:23.651832 12001 raft_consensus.cc:348] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } }
I20260925 11:05:23.652173 12001 raft_consensus.cc:374] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:23.652268 12001 raft_consensus.cc:729] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80c0bbaff0447e59f1a7b400194739a, State: Initialized, Role: FOLLOWER
I20260925 11:05:23.652761 12001 consensus_queue.cc:260] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } }
I20260925 11:05:23.653671 11980 heartbeater.cc:503] Master 127.24.159.254:46665 was elected leader, sending a full tablet report...
I20260925 11:05:23.654060 12001 ts_tablet_manager.cc:1428] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
W20260925 11:05:23.660432 11720 tablet.cc:2369] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:05:23.665863 12006 raft_consensus.cc:482] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:05:23.666121 12006 raft_consensus.cc:504] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } }
I20260925 11:05:23.667778 12006 leader_election.cc:288] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bc232fa0317409c9eca9e3d9750eec1 (127.24.159.193:43521), 86b18d1bb394426493fca2cdbbdc0ff8 (127.24.159.194:41465)
I20260925 11:05:23.673700 11982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.668105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43521 (local address 127.24.159.195:46717)
0925 11:05:23.668278 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.668296 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:23.668415 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:23.668902 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:23.668907 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:23.668938 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:23.669159 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.669171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.670358 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.670362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.672050 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.672065 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.673156 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.673160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.673292 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.673316 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 11:05:23.673382 (+ 66us) client_negotiation.cc:240] Negotiation successful
0925 11:05:23.673476 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 11:05:23.673815 11993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.668087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:46717 (local address 127.24.159.193:43521)
0925 11:05:23.668281 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.668287 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.668345 (+ 58us) server_negotiation.cc:407] Connection header received
0925 11:05:23.668534 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.668540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.668631 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.668720 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:23.669305 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.670235 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.672199 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.673013 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.673154 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.673223 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.673435 (+ 212us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.673564 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260925 11:05:23.674819 12007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.668442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:41465 (local address 127.24.159.195:37127)
0925 11:05:23.669136 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.669153 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:23.669270 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:23.669580 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:23.669583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:23.669600 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:23.669864 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.669871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.671064 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.671070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.672735 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:23.672746 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.674261 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.674269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:23.674430 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.674454 (+ 24us) client_negotiation.cc:769] Sending connection context
0925 11:05:23.674526 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 11:05:23.674609 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":342,"threads_started":1}
I20260925 11:05:23.674958 11995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.668767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:37127 (local address 127.24.159.194:41465)
0925 11:05:23.668955 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.668961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.669219 (+ 258us) server_negotiation.cc:407] Connection header received
0925 11:05:23.669408 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.669413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.669488 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.669576 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:23.669992 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.670917 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.672925 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.673972 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.674161 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.674246 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.674577 (+ 331us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.674700 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:05:23.674820 11674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a777fbd40bc145b9b1cfc955970f1337" candidate_uuid: "c80c0bbaff0447e59f1a7b400194739a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bc232fa0317409c9eca9e3d9750eec1" is_pre_election: true
I20260925 11:05:23.675418 11674 raft_consensus.cc:2454] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c80c0bbaff0447e59f1a7b400194739a in term 0.
I20260925 11:05:23.675756 11805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a777fbd40bc145b9b1cfc955970f1337" candidate_uuid: "c80c0bbaff0447e59f1a7b400194739a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" is_pre_election: true
I20260925 11:05:23.676075 11869 leader_election.cc:302] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bc232fa0317409c9eca9e3d9750eec1, c80c0bbaff0447e59f1a7b400194739a; no voters:
I20260925 11:05:23.676272 11805 raft_consensus.cc:2454] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c80c0bbaff0447e59f1a7b400194739a in term 0.
I20260925 11:05:23.676462 12006 raft_consensus.cc:2781] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:05:23.676604 12006 raft_consensus.cc:482] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:05:23.676713 12006 raft_consensus.cc:3037] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:23.678927 12006 raft_consensus.cc:504] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } }
I20260925 11:05:23.679517 12006 leader_election.cc:288] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [CANDIDATE]: Term 1 election: Requested vote from peers 2bc232fa0317409c9eca9e3d9750eec1 (127.24.159.193:43521), 86b18d1bb394426493fca2cdbbdc0ff8 (127.24.159.194:41465)
I20260925 11:05:23.679994 11805 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a777fbd40bc145b9b1cfc955970f1337" candidate_uuid: "c80c0bbaff0447e59f1a7b400194739a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86b18d1bb394426493fca2cdbbdc0ff8"
I20260925 11:05:23.679992 11674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a777fbd40bc145b9b1cfc955970f1337" candidate_uuid: "c80c0bbaff0447e59f1a7b400194739a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bc232fa0317409c9eca9e3d9750eec1"
I20260925 11:05:23.680220 11805 raft_consensus.cc:3037] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:23.680220 11674 raft_consensus.cc:3037] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:23.683372 11805 raft_consensus.cc:2454] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c80c0bbaff0447e59f1a7b400194739a in term 1.
I20260925 11:05:23.683372 11674 raft_consensus.cc:2454] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c80c0bbaff0447e59f1a7b400194739a in term 1.
I20260925 11:05:23.683753 11869 leader_election.cc:302] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bc232fa0317409c9eca9e3d9750eec1, c80c0bbaff0447e59f1a7b400194739a; no voters:
I20260925 11:05:23.684013 12006 raft_consensus.cc:2781] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:23.684422 12006 raft_consensus.cc:686] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 1 LEADER]: Becoming Leader. State: Replica: c80c0bbaff0447e59f1a7b400194739a, State: Running, Role: LEADER
I20260925 11:05:23.684923 12006 consensus_queue.cc:237] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } }
I20260925 11:05:23.689881 11532 catalog_manager.cc:5495] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a reported cstate change: term changed from 0 to 1, leader changed from <none> to c80c0bbaff0447e59f1a7b400194739a (127.24.159.195). New cstate: current_term: 1 leader_uuid: "c80c0bbaff0447e59f1a7b400194739a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c80c0bbaff0447e59f1a7b400194739a" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 36813 } health_report { overall_health: HEALTHY } } }
I20260925 11:05:23.721837 11997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.716943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40726 (local address 127.24.159.195:36813)
0925 11:05:23.717150 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.717156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.717177 (+ 21us) server_negotiation.cc:407] Connection header received
0925 11:05:23.717282 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.717286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.717363 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.717490 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:05:23.717781 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.718559 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.719881 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.720318 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.720430 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.720882 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:05:23.721393 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:05:23.721527 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.721602 (+ 75us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.721640 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 11:05:23.730084 11674 raft_consensus.cc:1261] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [term 1 FOLLOWER]: Refusing update from remote peer c80c0bbaff0447e59f1a7b400194739a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:23.730089 11805 raft_consensus.cc:1261] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [term 1 FOLLOWER]: Refusing update from remote peer c80c0bbaff0447e59f1a7b400194739a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:23.731071 12008 consensus_queue.cc:1035] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bc232fa0317409c9eca9e3d9750eec1" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:23.731406 12006 consensus_queue.cc:1035] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [LEADER]: Connected to new peer: Peer: permanent_uuid: "86b18d1bb394426493fca2cdbbdc0ff8" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:23.762639 11532 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:23.762878 11532 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:23.766325 11532 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e32e238fafce4a6292937808b83db035: Sending DeleteTablet for 3 replicas of tablet a777fbd40bc145b9b1cfc955970f1337
I20260925 11:05:23.767467 11654 tablet_service.cc:1514] Processing DeleteTablet for tablet a777fbd40bc145b9b1cfc955970f1337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:23 UTC) from {username='slave'} at 127.0.0.1:49268
I20260925 11:05:23.767619 11785 tablet_service.cc:1514] Processing DeleteTablet for tablet a777fbd40bc145b9b1cfc955970f1337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:23 UTC) from {username='slave'} at 127.0.0.1:33076
I20260925 11:05:23.768532 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 11592
I20260925 11:05:23.768476 11915 tablet_service.cc:1514] Processing DeleteTablet for tablet a777fbd40bc145b9b1cfc955970f1337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:23 UTC) from {username='slave'} at 127.0.0.1:40716
I20260925 11:05:23.783008 12023 tablet_replica.cc:330] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: stopping tablet replica
I20260925 11:05:23.783569 12023 raft_consensus.cc:2229] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:05:23.784385 12023 raft_consensus.cc:2258] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:23.788496 12021 tablet_replica.cc:330] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1: stopping tablet replica
I20260925 11:05:23.788782 12022 tablet_replica.cc:330] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: stopping tablet replica
I20260925 11:05:23.789223 12022 raft_consensus.cc:2229] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:23.789206 12021 raft_consensus.cc:2229] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:23.789736 12022 raft_consensus.cc:2258] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:23.789770 12021 raft_consensus.cc:2258] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:23.792085 12021 ts_tablet_manager.cc:1905] T a777fbd40bc145b9b1cfc955970f1337 P 2bc232fa0317409c9eca9e3d9750eec1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:23.792171 12023 ts_tablet_manager.cc:1905] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:23.792611 11993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.769955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49278 (local address 127.24.159.193:43521)
0925 11:05:23.770141 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.770147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.781054 (+ 10907us) server_negotiation.cc:407] Connection header received
0925 11:05:23.781548 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.781555 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.781690 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.781769 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:23.784850 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.786108 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.788486 (+ 2378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.789168 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.789277 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.790966 (+ 1689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:23.790976 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:23.790993 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:23.791737 (+ 744us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:23.791797 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:23.791823 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:23.791923 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.792153 (+ 230us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.792267 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:05:23.793649 11714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:23.794776 12022 ts_tablet_manager.cc:1905] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:23.799139 12023 ts_tablet_manager.cc:1918] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:23.799302 12023 log.cc:1196] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/a777fbd40bc145b9b1cfc955970f1337
I20260925 11:05:23.799813 12023 ts_tablet_manager.cc:1939] T a777fbd40bc145b9b1cfc955970f1337 P c80c0bbaff0447e59f1a7b400194739a: Deleting consensus metadata
I20260925 11:05:23.801156 12022 ts_tablet_manager.cc:1918] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:23.801367 12022 log.cc:1196] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/a777fbd40bc145b9b1cfc955970f1337
I20260925 11:05:23.801549 11518 catalog_manager.cc:4841] TS c80c0bbaff0447e59f1a7b400194739a (127.24.159.195:36813): tablet a777fbd40bc145b9b1cfc955970f1337 (table test-table [id=90b0a40d84e34800a92d031a18dcee87]) successfully deleted
I20260925 11:05:23.801841 12022 ts_tablet_manager.cc:1939] T a777fbd40bc145b9b1cfc955970f1337 P 86b18d1bb394426493fca2cdbbdc0ff8: Deleting consensus metadata
I20260925 11:05:23.803382 11518 catalog_manager.cc:4841] TS 86b18d1bb394426493fca2cdbbdc0ff8 (127.24.159.194:41465): tablet a777fbd40bc145b9b1cfc955970f1337 (table test-table [id=90b0a40d84e34800a92d031a18dcee87]) successfully deleted
I20260925 11:05:23.889765 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 11592
W20260925 11:05:23.909788 11518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.159.193:43521 (error 108)
I20260925 11:05:23.910243 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 11723
I20260925 11:05:23.912312 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.911750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43521 (local address 127.0.0.1:49294)
0925 11:05:23.912006 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.912132 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260925 11:05:23.912567 11518 catalog_manager.cc:4584] TS 2bc232fa0317409c9eca9e3d9750eec1 (127.24.159.193:43521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a777fbd40bc145b9b1cfc955970f1337: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43521: connect: Connection refused (error 111)
I20260925 11:05:23.917032 11995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.911104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.24.159.194:41465)
0925 11:05:23.911269 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:23.911274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:23.911368 (+ 94us) server_negotiation.cc:407] Connection header received
0925 11:05:23.911553 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:23.911557 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:23.911656 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:23.911754 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:23.912164 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.912973 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.914239 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:23.914647 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:23.914759 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:23.915737 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:23.915746 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:23.915760 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:23.916315 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:23.916362 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:23.916382 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:23.916480 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:23.916605 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 11:05:23.916720 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:05:23.917795 11845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:23.966351 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.965744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43521 (local address 127.0.0.1:49300)
0925 11:05:23.965989 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.966115 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260925 11:05:23.967556 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:23.967161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43521 (local address 127.0.0.1:49310)
0925 11:05:23.967324 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:23.967410 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 11:05:24.006635 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 11723
I20260925 11:05:24.020799 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 11853
I20260925 11:05:24.028081 11997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:24.021524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40730 (local address 127.24.159.195:36813)
0925 11:05:24.021715 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:24.021721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:24.021826 (+ 105us) server_negotiation.cc:407] Connection header received
0925 11:05:24.022005 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:24.022011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:24.022124 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:24.022247 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:24.022643 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:24.023419 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:24.024731 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:24.025161 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:24.025274 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:24.026322 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:24.026334 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:24.026355 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:24.027002 (+ 647us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:24.027053 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:24.027073 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:24.027174 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:24.027595 (+ 421us) server_negotiation.cc:299] Negotiation successful
0925 11:05:24.027703 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:05:24.028592 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:24.027918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43521 (local address 127.0.0.1:49324)
0925 11:05:24.028133 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:24.028397 (+ 264us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 11:05:24.029965 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:24.029606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43521 (local address 127.0.0.1:49326)
0925 11:05:24.029753 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:24.029836 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 11:05:24.030393 11975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:24.111783 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:24.111222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43521 (local address 127.0.0.1:49328)
0925 11:05:24.111463 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:24.111583 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260925 11:05:24.112989 11994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:24.112487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43521 (local address 127.0.0.1:49344)
0925 11:05:24.112718 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:24.112829 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260925 11:05:24.157927 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 11853
I20260925 11:05:24.175057 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 11502
I20260925 11:05:24.176195 11563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:24.283466 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 11502
2026-09-25T11:05:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2624 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:24 dist-test-slave-trms krb5kdc[12032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:24 dist-test-slave-trms krb5kdc[12032](info): set up 2 sockets
Sep 25 11:05:24 dist-test-slave-trms krb5kdc[12032](info): commencing operation
krb5kdc: starting...
W20260925 11:05:25.294445 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:25 dist-test-slave-trms krb5kdc[12032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:05:25.527273 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45291
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:45291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:25.845103 12045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:25.845398 12045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:25.845494 12045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:25.852730 12045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:25.852840 12045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:25.852885 12045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:25.852938 12045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:25.852979 12045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:25.862861 12045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:45291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:25.864109 12045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:25.865360 12045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:25.876770 12053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:25.876832 12050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:25.876801 12051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:25.877974 12045 server_base.cc:1034] running on GCE node
I20260925 11:05:25.878844 12045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:25.880299 12045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:25.881508 12045 hybrid_clock.cc:648] HybridClock initialized: now 1790334325881454 us; error 48 us; skew 500 ppm
I20260925 11:05:25.881850 12045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:25 dist-test-slave-trms krb5kdc[12032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334325, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:25.884812 12045 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:05:25.886293 12045 webserver.cc:460] Webserver started at http://127.24.159.254:46807/ using document root <none> and password file <none>
I20260925 11:05:25.886955 12045 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:25.887068 12045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:25.887356 12045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:25.890367 12045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "afa9b82608624d7e866928446c970d06"
format_stamp: "Formatted at 2026-09-25 11:05:25 on dist-test-slave-trms"
server_key: "54e76fea03d52320d22ab256e171460c"
server_key_iv: "51c062154d3c867f9d4cb29e3f7cfab3"
server_key_version: "encryptionkey@0"
I20260925 11:05:25.891108 12045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "afa9b82608624d7e866928446c970d06"
format_stamp: "Formatted at 2026-09-25 11:05:25 on dist-test-slave-trms"
server_key: "54e76fea03d52320d22ab256e171460c"
server_key_iv: "51c062154d3c867f9d4cb29e3f7cfab3"
server_key_version: "encryptionkey@0"
I20260925 11:05:25.896116 12045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:05:25.899510 12060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:25.900753 12045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:05:25.900909 12045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "afa9b82608624d7e866928446c970d06"
format_stamp: "Formatted at 2026-09-25 11:05:25 on dist-test-slave-trms"
server_key: "54e76fea03d52320d22ab256e171460c"
server_key_iv: "51c062154d3c867f9d4cb29e3f7cfab3"
server_key_version: "encryptionkey@0"
I20260925 11:05:25.901082 12045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:25.913961 12045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:25.917867 12045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:25.918120 12045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:25.934721 12045 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:45291
I20260925 11:05:25.934743 12111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:45291 every 8 connection(s)
I20260925 11:05:25.936092 12045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:25.939628 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 12045
I20260925 11:05:25.939707 12112 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:25.939803 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:25.940191 25215 external_mini_cluster.cc:1428] Setting key 7ecd45c029ff090af800987ccb5b6c26
I20260925 11:05:25.953673 12112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06: Bootstrap starting.
Sep 25 11:05:25 dist-test-slave-trms krb5kdc[12032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:25.957439 12112 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:25.958731 12112 log.cc:824] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:25.961937 12112 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06: No bootstrap required, opened a new log
I20260925 11:05:25.962991 12115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:25.941817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51440 (local address 127.24.159.254:45291)
0925 11:05:25.942675 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:25.942689 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:25.942742 (+ 53us) server_negotiation.cc:407] Connection header received
0925 11:05:25.944441 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:25.944470 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:25.945012 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:25.945447 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:25.946677 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:25.948635 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:25.949913 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:25.950987 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:25.951181 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:25.955000 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:25.955011 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:25.955028 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:25.955085 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:25.958763 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:25.959470 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:25.959476 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:25.959482 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:25.959653 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:25.960046 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:25.960051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:25.960054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:25.960717 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:25.960983 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:25.961290 (+ 307us) server_negotiation.cc:299] Negotiation successful
0925 11:05:25.961700 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":347,"threads_started":1}
I20260925 11:05:25.969446 12112 raft_consensus.cc:348] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa9b82608624d7e866928446c970d06" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45291 } }
I20260925 11:05:25.969775 12112 raft_consensus.cc:374] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:25.969846 12112 raft_consensus.cc:729] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afa9b82608624d7e866928446c970d06, State: Initialized, Role: FOLLOWER
I20260925 11:05:25.970382 12112 consensus_queue.cc:260] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa9b82608624d7e866928446c970d06" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45291 } }
I20260925 11:05:25.970588 12112 raft_consensus.cc:388] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:25.970683 12112 raft_consensus.cc:482] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:25.970803 12112 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:25.972815 12112 raft_consensus.cc:504] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa9b82608624d7e866928446c970d06" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45291 } }
I20260925 11:05:25.973290 12112 leader_election.cc:302] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afa9b82608624d7e866928446c970d06; no voters:
I20260925 11:05:25.973888 12112 leader_election.cc:288] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:25.973985 12117 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:25.974581 12117 raft_consensus.cc:686] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [term 1 LEADER]: Becoming Leader. State: Replica: afa9b82608624d7e866928446c970d06, State: Running, Role: LEADER
I20260925 11:05:25.975107 12117 consensus_queue.cc:237] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa9b82608624d7e866928446c970d06" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45291 } }
I20260925 11:05:25.976064 12112 sys_catalog.cc:564] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:25.977859 12118 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afa9b82608624d7e866928446c970d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa9b82608624d7e866928446c970d06" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45291 } } }
I20260925 11:05:25.978231 12118 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:25.980152 12119 sys_catalog.cc:455] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afa9b82608624d7e866928446c970d06. Latest consensus state: current_term: 1 leader_uuid: "afa9b82608624d7e866928446c970d06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afa9b82608624d7e866928446c970d06" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45291 } } }
I20260925 11:05:25.980528 12119 sys_catalog.cc:458] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:25.981469 12123 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:25.986563 12123 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:25.994812 12123 catalog_manager.cc:1347] Generated new cluster ID: 5609f61bd45446d0ac219a32d6e2c6ed
I20260925 11:05:25.994911 12123 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:26.012722 12123 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:26.013976 12123 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:26.023659 12123 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06: Generated new TSK 0
I20260925 11:05:26.024528 12123 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.159.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
Entry for principal kudu/127.24.159.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab.
I20260925 11:05:26.083454 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45291
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:26.417320 12138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:26.417598 12138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:26.417749 12138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:26.425151 12138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:26.425271 12138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:26.425427 12138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:05:26.435228 12138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.193.keytab
--principal=kudu/127.24.159.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:26.436507 12138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:26.437847 12138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:26.450537 12143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:26.450495 12144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:26.450656 12146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:26.451529 12138 server_base.cc:1034] running on GCE node
I20260925 11:05:26.452142 12138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:26.453105 12138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:26.454280 12138 hybrid_clock.cc:648] HybridClock initialized: now 1790334326454221 us; error 50 us; skew 500 ppm
I20260925 11:05:26.454593 12138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:26 dist-test-slave-trms krb5kdc[12032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:26.457502 12138 init.cc:377] Logged in from keytab as kudu/127.24.159.193@KRBTEST.COM (short username kudu)
I20260925 11:05:26.458966 12138 webserver.cc:460] Webserver started at http://127.24.159.193:38445/ using document root <none> and password file <none>
I20260925 11:05:26.459618 12138 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:26.459755 12138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:26.460044 12138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:26.462926 12138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "365b58ae62c84922852df34b37ac7d52"
format_stamp: "Formatted at 2026-09-25 11:05:26 on dist-test-slave-trms"
server_key: "169d8aaa18406af011e0189baa58066e"
server_key_iv: "7ef6bb597702e593be859c7c3ed777da"
server_key_version: "encryptionkey@0"
I20260925 11:05:26.463733 12138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "365b58ae62c84922852df34b37ac7d52"
format_stamp: "Formatted at 2026-09-25 11:05:26 on dist-test-slave-trms"
server_key: "169d8aaa18406af011e0189baa58066e"
server_key_iv: "7ef6bb597702e593be859c7c3ed777da"
server_key_version: "encryptionkey@0"
I20260925 11:05:26.468847 12138 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:05:26.472322 12153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:26.473680 12138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:05:26.473811 12138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "365b58ae62c84922852df34b37ac7d52"
format_stamp: "Formatted at 2026-09-25 11:05:26 on dist-test-slave-trms"
server_key: "169d8aaa18406af011e0189baa58066e"
server_key_iv: "7ef6bb597702e593be859c7c3ed777da"
server_key_version: "encryptionkey@0"
I20260925 11:05:26.473984 12138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:26.488217 12138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:26.492002 12138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:26.492244 12138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:26.493110 12138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:26.494393 12138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:26.494463 12138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:26.494549 12138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:26.494593 12138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:26.523927 12138 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:34333
I20260925 11:05:26.524015 12265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:34333 every 8 connection(s)
I20260925 11:05:26.525214 12138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:05:26.531908 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 12138
I20260925 11:05:26.532058 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:05:26.532495 25215 external_mini_cluster.cc:1428] Setting key 3cb7a080326a40da3bca32b180722c44
Sep 25 11:05:26 dist-test-slave-trms krb5kdc[12032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.193@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:26.544806 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:26.527832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:53209 (local address 127.24.159.254:45291)
0925 11:05:26.528267 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:26.528273 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:26.529311 (+ 1038us) server_negotiation.cc:407] Connection header received
0925 11:05:26.531162 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:26.531167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:26.531236 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:26.531331 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:26.532947 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:26.533671 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:26.535266 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:26.535669 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:26.535780 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:26.539254 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:26.539259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:26.539262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:26.539295 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:26.541741 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:26.542515 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:26.542519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:26.542521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:26.542641 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:26.543051 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:26.543056 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:26.543059 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:26.543358 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:26.543489 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:26.544224 (+ 735us) server_negotiation.cc:299] Negotiation successful
0925 11:05:26.544451 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":193,"threads_started":1}
I20260925 11:05:26.545790 12269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:26.528450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:45291 (local address 127.24.159.193:53209)
0925 11:05:26.529154 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:26.529197 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:26.530889 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:26.531572 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:26.531580 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:26.532159 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:26.532773 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:26.532794 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:26.533821 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:26.533826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:26.535129 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:26.535142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:26.535881 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:26.535888 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:26.536118 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:26.536726 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:05:26.536750 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:26.539037 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:26.541885 (+ 2848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:26.541894 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:26.541912 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:26.542396 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:26.542740 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:26.542745 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:26.542749 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:26.542937 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:26.543477 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:26.543485 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:26.543855 (+ 370us) client_negotiation.cc:769] Sending connection context
0925 11:05:26.544083 (+ 228us) client_negotiation.cc:240] Negotiation successful
0925 11:05:26.544610 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":263,"threads_started":1}
I20260925 11:05:26.547336 12266 heartbeater.cc:347] Connected to a master server at 127.24.159.254:45291
I20260925 11:05:26.547679 12266 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:26.548552 12266 heartbeater.cc:511] Master 127.24.159.254:45291 requested a full tablet report, sending...
I20260925 11:05:26.550932 12077 ts_manager.cc:194] Registered new tserver with Master: 365b58ae62c84922852df34b37ac7d52 (127.24.159.193:34333)
I20260925 11:05:26.552383 12077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.193@KRBTEST.COM'} at 127.24.159.193:53209
WARNING: no policy specified for kudu/127.24.159.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
Entry for principal kudu/127.24.159.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab.
I20260925 11:05:26.584988 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45291
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:26.903277 12272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:26.903591 12272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:26.903740 12272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:26.911250 12272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:26.911381 12272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:26.911540 12272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:05:26.921393 12272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.194.keytab
--principal=kudu/127.24.159.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:26.922704 12272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:26.923977 12272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:26.936830 12277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:26.936856 12278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:26.937464 12272 server_base.cc:1034] running on GCE node
W20260925 11:05:26.937676 12280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:26.938365 12272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:26.939281 12272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:26.940470 12272 hybrid_clock.cc:648] HybridClock initialized: now 1790334326940411 us; error 52 us; skew 500 ppm
I20260925 11:05:26.940785 12272 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:26 dist-test-slave-trms krb5kdc[12032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:26.943897 12272 init.cc:377] Logged in from keytab as kudu/127.24.159.194@KRBTEST.COM (short username kudu)
I20260925 11:05:26.950726 12272 webserver.cc:460] Webserver started at http://127.24.159.194:35155/ using document root <none> and password file <none>
I20260925 11:05:26.951428 12272 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:26.951545 12272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:26.951855 12272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:26.954770 12272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "b58363c272c8446da47c12f8869fd32d"
format_stamp: "Formatted at 2026-09-25 11:05:26 on dist-test-slave-trms"
server_key: "f61c70dbaf5b95a29de8568fdf171d3b"
server_key_iv: "f256778b0fb52e7b39b0420bdf0df156"
server_key_version: "encryptionkey@0"
I20260925 11:05:26.955518 12272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "b58363c272c8446da47c12f8869fd32d"
format_stamp: "Formatted at 2026-09-25 11:05:26 on dist-test-slave-trms"
server_key: "f61c70dbaf5b95a29de8568fdf171d3b"
server_key_iv: "f256778b0fb52e7b39b0420bdf0df156"
server_key_version: "encryptionkey@0"
I20260925 11:05:26.960832 12272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:05:26.964308 12287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:26.965605 12272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260925 11:05:26.965757 12272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "b58363c272c8446da47c12f8869fd32d"
format_stamp: "Formatted at 2026-09-25 11:05:26 on dist-test-slave-trms"
server_key: "f61c70dbaf5b95a29de8568fdf171d3b"
server_key_iv: "f256778b0fb52e7b39b0420bdf0df156"
server_key_version: "encryptionkey@0"
I20260925 11:05:26.965950 12272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:26.979724 12272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:26.983948 12272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:26.984223 12272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:26.985222 12272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:26.986608 12272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:26.986681 12272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:26.986779 12272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:26.986840 12272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:27.017258 12272 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:46507
I20260925 11:05:27.017303 12399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:46507 every 8 connection(s)
I20260925 11:05:27.018568 12272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:05:27.027580 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 12272
I20260925 11:05:27.027766 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:05:27.028292 25215 external_mini_cluster.cc:1428] Setting key dc365af18571bf88b7c27ca5f53d3711
Sep 25 11:05:27 dist-test-slave-trms krb5kdc[12032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334326, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.194@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:27.039353 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.021125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:33981 (local address 127.24.159.254:45291)
0925 11:05:27.021301 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.021306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.022542 (+ 1236us) server_negotiation.cc:407] Connection header received
0925 11:05:27.024478 (+ 1936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.024483 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.024567 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.024664 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:27.026210 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.027005 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.029163 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.029618 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.029733 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.033635 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:27.033640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:27.033643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:27.033676 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:27.036163 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.036970 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.036973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.036976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.037097 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.037523 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.037527 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.037528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.037819 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:27.037943 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.038776 (+ 833us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.038999 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260925 11:05:27.040323 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.021620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:45291 (local address 127.24.159.194:33981)
0925 11:05:27.022385 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:27.022425 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:27.024218 (+ 1793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:27.024897 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:27.024906 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:27.025439 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:27.026008 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.026025 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.027146 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.027150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.029008 (+ 1858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.029022 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.029841 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.029849 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.030064 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.030746 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:05:27.030775 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:27.033418 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:27.036313 (+ 2895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:27.036321 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:27.036339 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:27.036874 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:27.037194 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:27.037198 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:27.037201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:27.037434 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:27.037955 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:27.037963 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:27.038421 (+ 458us) client_negotiation.cc:769] Sending connection context
0925 11:05:27.038713 (+ 292us) client_negotiation.cc:240] Negotiation successful
0925 11:05:27.039190 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":291,"threads_started":1}
I20260925 11:05:27.041669 12400 heartbeater.cc:347] Connected to a master server at 127.24.159.254:45291
I20260925 11:05:27.041973 12400 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:27.042735 12400 heartbeater.cc:511] Master 127.24.159.254:45291 requested a full tablet report, sending...
I20260925 11:05:27.044164 12077 ts_manager.cc:194] Registered new tserver with Master: b58363c272c8446da47c12f8869fd32d (127.24.159.194:46507)
I20260925 11:05:27.044967 12077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.194@KRBTEST.COM'} at 127.24.159.194:33981
WARNING: no policy specified for kudu/127.24.159.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
Entry for principal kudu/127.24.159.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab.
I20260925 11:05:27.080973 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45291
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:27.400640 12405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:27.400913 12405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:27.401046 12405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:27.408293 12405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:27.408406 12405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:27.408552 12405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:05:27.419399 12405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.195.keytab
--principal=kudu/127.24.159.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:45291
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:27.420742 12405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:27.422071 12405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:27.434516 12411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:27.434578 12410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:27.434609 12413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:27.435357 12405 server_base.cc:1034] running on GCE node
I20260925 11:05:27.435891 12405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:27.436749 12405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:27.437932 12405 hybrid_clock.cc:648] HybridClock initialized: now 1790334327437876 us; error 47 us; skew 500 ppm
I20260925 11:05:27.438226 12405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:27 dist-test-slave-trms krb5kdc[12032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334327, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:27.441097 12405 init.cc:377] Logged in from keytab as kudu/127.24.159.195@KRBTEST.COM (short username kudu)
I20260925 11:05:27.442462 12405 webserver.cc:460] Webserver started at http://127.24.159.195:37241/ using document root <none> and password file <none>
I20260925 11:05:27.443078 12405 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:27.443176 12405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:27.443437 12405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:27.446250 12405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "96a44596d538406fb8357cfdc8c37221"
format_stamp: "Formatted at 2026-09-25 11:05:27 on dist-test-slave-trms"
server_key: "ee7a5bbb1d1d5e129e77dc9e72c4a221"
server_key_iv: "61a4d79d596fdb331d7624919966ccda"
server_key_version: "encryptionkey@0"
I20260925 11:05:27.447054 12405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "96a44596d538406fb8357cfdc8c37221"
format_stamp: "Formatted at 2026-09-25 11:05:27 on dist-test-slave-trms"
server_key: "ee7a5bbb1d1d5e129e77dc9e72c4a221"
server_key_iv: "61a4d79d596fdb331d7624919966ccda"
server_key_version: "encryptionkey@0"
I20260925 11:05:27.451896 12405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260925 11:05:27.455080 12420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:27.456323 12405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260925 11:05:27.456463 12405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "96a44596d538406fb8357cfdc8c37221"
format_stamp: "Formatted at 2026-09-25 11:05:27 on dist-test-slave-trms"
server_key: "ee7a5bbb1d1d5e129e77dc9e72c4a221"
server_key_iv: "61a4d79d596fdb331d7624919966ccda"
server_key_version: "encryptionkey@0"
I20260925 11:05:27.456629 12405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:27.479297 12405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:27.483635 12405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:27.483899 12405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:27.484853 12405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:27.486129 12405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:27.486231 12405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:27.486363 12405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:27.486413 12405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:27.515086 12405 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:38137
I20260925 11:05:27.515123 12532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:38137 every 8 connection(s)
I20260925 11:05:27.516402 12405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:05:27.522855 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 12405
I20260925 11:05:27.523041 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:05:27.523566 25215 external_mini_cluster.cc:1428] Setting key c450719137377438b45df6b458ee880b
Sep 25 11:05:27 dist-test-slave-trms krb5kdc[12032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334327, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.195@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:27.536419 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.518891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:42601 (local address 127.24.159.254:45291)
0925 11:05:27.519069 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.519073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.520289 (+ 1216us) server_negotiation.cc:407] Connection header received
0925 11:05:27.522142 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.522148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.522279 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.522380 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:27.524427 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.525639 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.527556 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.528173 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.528298 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.531494 (+ 3196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:27.531499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:27.531502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:27.531537 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:27.533781 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.534477 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.534481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.534483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.534611 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.534945 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.534948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.534950 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.535208 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:27.535320 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.535981 (+ 661us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.536159 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:05:27.537297 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.519425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:45291 (local address 127.24.159.195:42601)
0925 11:05:27.520135 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:27.520175 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:27.521835 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:27.522671 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:27.522683 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:27.523459 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:27.524226 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.524253 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.525817 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.525823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.527426 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.527440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.528358 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.528362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.528552 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.529160 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:05:27.529183 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:27.531286 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:27.533909 (+ 2623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:27.533916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:27.533927 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:27.534396 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:27.534689 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:05:27.534692 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:05:27.534694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:05:27.534866 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:05:27.535304 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:27.535316 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:27.535607 (+ 291us) client_negotiation.cc:769] Sending connection context
0925 11:05:27.535821 (+ 214us) client_negotiation.cc:240] Negotiation successful
0925 11:05:27.536187 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":264,"threads_started":1}
I20260925 11:05:27.538704 12533 heartbeater.cc:347] Connected to a master server at 127.24.159.254:45291
I20260925 11:05:27.539016 12533 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:27.539680 12533 heartbeater.cc:511] Master 127.24.159.254:45291 requested a full tablet report, sending...
I20260925 11:05:27.540964 12077 ts_manager.cc:194] Registered new tserver with Master: 96a44596d538406fb8357cfdc8c37221 (127.24.159.195:38137)
I20260925 11:05:27.541801 12077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.159.195@KRBTEST.COM'} at 127.24.159.195:42601
I20260925 11:05:27.551050 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:05:27.555944 12266 heartbeater.cc:503] Master 127.24.159.254:45291 was elected leader, sending a full tablet report...
I20260925 11:05:27.566572 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.554802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51444 (local address 127.24.159.254:45291)
0925 11:05:27.554955 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.554961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.555367 (+ 406us) server_negotiation.cc:407] Connection header received
0925 11:05:27.555523 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.555528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.555588 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.555665 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:27.557067 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.557855 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.559137 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.559621 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.559737 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.561877 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:27.561882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:27.561885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:27.561915 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:27.564113 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.564741 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.564745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.564747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.564876 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.565213 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.565218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.565222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.565517 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:27.565750 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.566048 (+ 298us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.566288 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:05:27.569983 12077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:05:27.576622 12077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:05:27.599130 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.588929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.24.159.195:38137)
0925 11:05:27.591079 (+ 2150us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.591089 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.591126 (+ 37us) server_negotiation.cc:407] Connection header received
0925 11:05:27.591238 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.591247 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.591535 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.592418 (+ 883us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:27.592723 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.593973 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.596749 (+ 2776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.597924 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.598120 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.598358 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.598610 (+ 252us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.598784 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1959,"thread_start_us":1794,"threads_started":1}
I20260925 11:05:27.599120 12545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.588636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:38137 (local address 127.0.0.1:40542)
0925 11:05:27.589571 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:27.589608 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:27.589781 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:27.591791 (+ 2010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:27.591800 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:27.591847 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:27.592202 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.592215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.594130 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.594135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.596589 (+ 2454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.596602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.598143 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.598150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.598358 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.598414 (+ 56us) client_negotiation.cc:769] Sending connection context
0925 11:05:27.598552 (+ 138us) client_negotiation.cc:240] Negotiation successful
0925 11:05:27.598748 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"thread_start_us":576,"threads_started":1}
I20260925 11:05:27.600062 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.589000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:46507 (local address 127.0.0.1:52624)
0925 11:05:27.590373 (+ 1373us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:27.590390 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:27.590499 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:27.592398 (+ 1899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:27.592403 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:27.592428 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:27.592791 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.592804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.594567 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.594573 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.597209 (+ 2636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.597224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.599318 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.599326 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.599497 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.599528 (+ 31us) client_negotiation.cc:769] Sending connection context
0925 11:05:27.599616 (+ 88us) client_negotiation.cc:240] Negotiation successful
0925 11:05:27.599753 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1241,"thread_start_us":1081,"threads_started":1}
I20260925 11:05:27.600936 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.589598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.24.159.194:46507)
0925 11:05:27.591902 (+ 2304us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.591910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.591935 (+ 25us) server_negotiation.cc:407] Connection header received
0925 11:05:27.592025 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.592030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.592224 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.592396 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:27.592991 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.594436 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.597394 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.598774 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.598977 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.599144 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.600436 (+ 1292us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.600618 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2172,"thread_start_us":2067,"threads_started":1}
I20260925 11:05:27.601404 12549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.590500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:34333 (local address 127.0.0.1:60744)
0925 11:05:27.591312 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:27.591330 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:27.591451 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:27.593969 (+ 2518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:27.593973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:27.594000 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:27.594427 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.594440 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.596321 (+ 1881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.596327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.598060 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.598076 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.600674 (+ 2598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.600681 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.600840 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.600870 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 11:05:27.600953 (+ 83us) client_negotiation.cc:240] Negotiation successful
0925 11:05:27.601093 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"thread_start_us":556,"threads_started":1}
I20260925 11:05:27.601436 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.590810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60744 (local address 127.24.159.193:34333)
0925 11:05:27.593394 (+ 2584us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.593402 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.593428 (+ 26us) server_negotiation.cc:407] Connection header received
0925 11:05:27.593517 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.593525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.593763 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.593950 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:27.594619 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.596157 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.598277 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.600487 (+ 2210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.600677 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.600842 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.600994 (+ 152us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.601146 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2420,"thread_start_us":2270,"threads_started":1}
I20260925 11:05:27.603682 12468 tablet_service.cc:1467] Processing CreateTablet for tablet c52fb61129e04d3ab1b41b482ba9e22c (DEFAULT_TABLE table=test-table [id=aa102d571fda4bf59bbcf0668ea5c079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:27.604521 12335 tablet_service.cc:1467] Processing CreateTablet for tablet c52fb61129e04d3ab1b41b482ba9e22c (DEFAULT_TABLE table=test-table [id=aa102d571fda4bf59bbcf0668ea5c079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:27.605273 12468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52fb61129e04d3ab1b41b482ba9e22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:27.605693 12335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52fb61129e04d3ab1b41b482ba9e22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:27.605485 12201 tablet_service.cc:1467] Processing CreateTablet for tablet c52fb61129e04d3ab1b41b482ba9e22c (DEFAULT_TABLE table=test-table [id=aa102d571fda4bf59bbcf0668ea5c079]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:27.606920 12201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c52fb61129e04d3ab1b41b482ba9e22c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:27.617017 12551 tablet_bootstrap.cc:492] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: Bootstrap starting.
I20260925 11:05:27.619448 12552 tablet_bootstrap.cc:492] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: Bootstrap starting.
I20260925 11:05:27.620683 12551 tablet_bootstrap.cc:654] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:27.621011 12553 tablet_bootstrap.cc:492] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: Bootstrap starting.
I20260925 11:05:27.622500 12551 log.cc:824] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:27.623062 12552 tablet_bootstrap.cc:654] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:27.624166 12552 log.cc:824] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:27.624752 12553 tablet_bootstrap.cc:654] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:27.625587 12551 tablet_bootstrap.cc:492] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: No bootstrap required, opened a new log
I20260925 11:05:27.625871 12551 ts_tablet_manager.cc:1397] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260925 11:05:27.625914 12553 log.cc:824] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:27.627288 12552 tablet_bootstrap.cc:492] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: No bootstrap required, opened a new log
I20260925 11:05:27.627542 12552 ts_tablet_manager.cc:1397] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260925 11:05:27.628327 12553 tablet_bootstrap.cc:492] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: No bootstrap required, opened a new log
I20260925 11:05:27.628575 12553 ts_tablet_manager.cc:1397] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260925 11:05:27.635893 12553 raft_consensus.cc:348] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } }
I20260925 11:05:27.636297 12553 raft_consensus.cc:374] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:27.636400 12553 raft_consensus.cc:729] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365b58ae62c84922852df34b37ac7d52, State: Initialized, Role: FOLLOWER
I20260925 11:05:27.636240 12551 raft_consensus.cc:348] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } }
I20260925 11:05:27.636694 12551 raft_consensus.cc:374] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:27.636792 12551 raft_consensus.cc:729] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58363c272c8446da47c12f8869fd32d, State: Initialized, Role: FOLLOWER
I20260925 11:05:27.637029 12553 consensus_queue.cc:260] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } }
I20260925 11:05:27.637391 12551 consensus_queue.cc:260] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } }
I20260925 11:05:27.637737 12552 raft_consensus.cc:348] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } }
I20260925 11:05:27.638211 12552 raft_consensus.cc:374] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:27.638306 12552 raft_consensus.cc:729] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a44596d538406fb8357cfdc8c37221, State: Initialized, Role: FOLLOWER
I20260925 11:05:27.639132 12553 ts_tablet_manager.cc:1428] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260925 11:05:27.639021 12552 consensus_queue.cc:260] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } }
I20260925 11:05:27.643558 12533 heartbeater.cc:503] Master 127.24.159.254:45291 was elected leader, sending a full tablet report...
I20260925 11:05:27.644829 12400 heartbeater.cc:503] Master 127.24.159.254:45291 was elected leader, sending a full tablet report...
I20260925 11:05:27.645450 12551 ts_tablet_manager.cc:1428] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: Time spent starting tablet: real 0.019s user 0.016s sys 0.004s
I20260925 11:05:27.646415 12552 ts_tablet_manager.cc:1428] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: Time spent starting tablet: real 0.019s user 0.011s sys 0.007s
I20260925 11:05:27.704996 12557 raft_consensus.cc:482] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:05:27.705279 12557 raft_consensus.cc:504] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } }
I20260925 11:05:27.707041 12557 leader_election.cc:288] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 96a44596d538406fb8357cfdc8c37221 (127.24.159.195:38137), b58363c272c8446da47c12f8869fd32d (127.24.159.194:46507)
I20260925 11:05:27.715119 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.707488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:42033 (local address 127.24.159.195:38137)
0925 11:05:27.707837 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.707844 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.708534 (+ 690us) server_negotiation.cc:407] Connection header received
0925 11:05:27.708645 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.708650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.708717 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.708797 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:27.709424 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.710476 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.712850 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.714052 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.714230 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.714306 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.714728 (+ 422us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.714863 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260925 11:05:27.715119 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.707584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:56647 (local address 127.24.159.194:46507)
0925 11:05:27.708000 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.708006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.708234 (+ 228us) server_negotiation.cc:407] Connection header received
0925 11:05:27.708458 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.708463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.708535 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.708632 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:27.709193 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.710285 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.712879 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.714056 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.714228 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.714306 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.714734 (+ 428us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.714869 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260925 11:05:27.715299 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.707396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:38137 (local address 127.24.159.193:42033)
0925 11:05:27.708441 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:27.708457 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:27.708547 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:27.708835 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:27.708843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:27.708863 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:27.709289 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.709298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.710600 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.710605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.712700 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.712719 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.714348 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.714355 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.714540 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.714606 (+ 66us) client_negotiation.cc:769] Sending connection context
0925 11:05:27.714695 (+ 89us) client_negotiation.cc:240] Negotiation successful
0925 11:05:27.714845 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"mutex_wait_us":58,"thread_start_us":378,"threads_started":1}
I20260925 11:05:27.715662 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.707321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:46507 (local address 127.24.159.193:56647)
0925 11:05:27.708127 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:27.708147 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:27.708317 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:27.708670 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:27.708673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:27.708694 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:27.709039 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.709056 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.710415 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.710419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.712719 (+ 2300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:27.712729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.714328 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.714335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:27.714542 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.714601 (+ 59us) client_negotiation.cc:769] Sending connection context
0925 11:05:27.714692 (+ 91us) client_negotiation.cc:240] Negotiation successful
0925 11:05:27.714845 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"mutex_wait_us":75,"thread_start_us":837,"threads_started":1}
I20260925 11:05:27.716251 12488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52fb61129e04d3ab1b41b482ba9e22c" candidate_uuid: "365b58ae62c84922852df34b37ac7d52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a44596d538406fb8357cfdc8c37221" is_pre_election: true
I20260925 11:05:27.716602 12355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52fb61129e04d3ab1b41b482ba9e22c" candidate_uuid: "365b58ae62c84922852df34b37ac7d52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58363c272c8446da47c12f8869fd32d" is_pre_election: true
I20260925 11:05:27.716743 12488 raft_consensus.cc:2454] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365b58ae62c84922852df34b37ac7d52 in term 0.
I20260925 11:05:27.717056 12355 raft_consensus.cc:2454] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365b58ae62c84922852df34b37ac7d52 in term 0.
I20260925 11:05:27.717355 12154 leader_election.cc:302] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365b58ae62c84922852df34b37ac7d52, 96a44596d538406fb8357cfdc8c37221; no voters:
I20260925 11:05:27.717705 12557 raft_consensus.cc:2781] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:05:27.717841 12557 raft_consensus.cc:482] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:05:27.717940 12557 raft_consensus.cc:3037] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:27.720281 12557 raft_consensus.cc:504] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } }
I20260925 11:05:27.720885 12557 leader_election.cc:288] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [CANDIDATE]: Term 1 election: Requested vote from peers 96a44596d538406fb8357cfdc8c37221 (127.24.159.195:38137), b58363c272c8446da47c12f8869fd32d (127.24.159.194:46507)
I20260925 11:05:27.721351 12355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52fb61129e04d3ab1b41b482ba9e22c" candidate_uuid: "365b58ae62c84922852df34b37ac7d52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58363c272c8446da47c12f8869fd32d"
I20260925 11:05:27.721387 12488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c52fb61129e04d3ab1b41b482ba9e22c" candidate_uuid: "365b58ae62c84922852df34b37ac7d52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a44596d538406fb8357cfdc8c37221"
I20260925 11:05:27.721534 12355 raft_consensus.cc:3037] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:27.721568 12488 raft_consensus.cc:3037] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:27.724087 12488 raft_consensus.cc:2454] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365b58ae62c84922852df34b37ac7d52 in term 1.
I20260925 11:05:27.724090 12355 raft_consensus.cc:2454] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365b58ae62c84922852df34b37ac7d52 in term 1.
I20260925 11:05:27.724475 12154 leader_election.cc:302] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 365b58ae62c84922852df34b37ac7d52, 96a44596d538406fb8357cfdc8c37221, b58363c272c8446da47c12f8869fd32d; no voters:
I20260925 11:05:27.724776 12557 raft_consensus.cc:2781] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:27.725185 12557 raft_consensus.cc:686] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 1 LEADER]: Becoming Leader. State: Replica: 365b58ae62c84922852df34b37ac7d52, State: Running, Role: LEADER
I20260925 11:05:27.725696 12557 consensus_queue.cc:237] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } }
I20260925 11:05:27.730075 12077 catalog_manager.cc:5495] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 reported cstate change: term changed from 0 to 1, leader changed from <none> to 365b58ae62c84922852df34b37ac7d52 (127.24.159.193). New cstate: current_term: 1 leader_uuid: "365b58ae62c84922852df34b37ac7d52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "365b58ae62c84922852df34b37ac7d52" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 34333 } health_report { overall_health: HEALTHY } } }
I20260925 11:05:27.764094 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.759401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60748 (local address 127.24.159.193:34333)
0925 11:05:27.759569 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.759574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.759591 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:05:27.759688 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.759692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.759771 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.759927 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:05:27.760257 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.760944 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.762265 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.762654 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.762741 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.763152 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:05:27.763671 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:05:27.763780 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.763821 (+ 41us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.763882 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260925 11:05:27.768164 12534 tablet.cc:2369] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260925 11:05:27.770404 12401 tablet.cc:2369] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:05:27.772415 12488 raft_consensus.cc:1261] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [term 1 FOLLOWER]: Refusing update from remote peer 365b58ae62c84922852df34b37ac7d52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:27.772414 12355 raft_consensus.cc:1261] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [term 1 FOLLOWER]: Refusing update from remote peer 365b58ae62c84922852df34b37ac7d52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:27.773494 12557 consensus_queue.cc:1035] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a44596d538406fb8357cfdc8c37221" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 38137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:27.773795 12562 consensus_queue.cc:1035] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b58363c272c8446da47c12f8869fd32d" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 46507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260925 11:05:27.777653 12267 tablet.cc:2369] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:05:27.782809 12565 mvcc.cc:205] Tried to move back new op lower bound from 7333209406545612800 to 7333209406368493568. Current Snapshot: MvccSnapshot[applied={T|T < 7333209406545612800}]
I20260925 11:05:27.795578 12567 mvcc.cc:205] Tried to move back new op lower bound from 7333209406545612800 to 7333209406368493568. Current Snapshot: MvccSnapshot[applied={T|T < 7333209406545612800}]
I20260925 11:05:27.808324 12076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:27.808663 12076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:27.812578 12076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afa9b82608624d7e866928446c970d06: Sending DeleteTablet for 3 replicas of tablet c52fb61129e04d3ab1b41b482ba9e22c
I20260925 11:05:27.813687 12468 tablet_service.cc:1514] Processing DeleteTablet for tablet c52fb61129e04d3ab1b41b482ba9e22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:27 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:40542
I20260925 11:05:27.813786 12201 tablet_service.cc:1514] Processing DeleteTablet for tablet c52fb61129e04d3ab1b41b482ba9e22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:27 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:60744
I20260925 11:05:27.814419 12335 tablet_service.cc:1514] Processing DeleteTablet for tablet c52fb61129e04d3ab1b41b482ba9e22c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:27 UTC) from {username='kudu', principal='kudu/127.24.159.254@KRBTEST.COM'} at 127.0.0.1:52624
I20260925 11:05:27.815274 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 12138
I20260925 11:05:27.823880 12576 tablet_replica.cc:330] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: stopping tablet replica
I20260925 11:05:27.824502 12576 raft_consensus.cc:2229] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:27.825515 12576 raft_consensus.cc:2258] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:27.826670 12575 tablet_replica.cc:330] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: stopping tablet replica
I20260925 11:05:27.827132 12575 raft_consensus.cc:2229] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:05:27.827826 12575 raft_consensus.cc:2258] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:27.827965 12576 ts_tablet_manager.cc:1905] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:27.831089 12574 tablet_replica.cc:330] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: stopping tablet replica
I20260925 11:05:27.831532 12574 raft_consensus.cc:2229] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:27.832059 12574 raft_consensus.cc:2258] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:27.833725 12575 ts_tablet_manager.cc:1905] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:27.834335 12574 ts_tablet_manager.cc:1905] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:27.836230 12576 ts_tablet_manager.cc:1918] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:27.836432 12576 log.cc:1196] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/c52fb61129e04d3ab1b41b482ba9e22c
I20260925 11:05:27.837034 12576 ts_tablet_manager.cc:1939] T c52fb61129e04d3ab1b41b482ba9e22c P b58363c272c8446da47c12f8869fd32d: Deleting consensus metadata
I20260925 11:05:27.839105 12064 catalog_manager.cc:4841] TS b58363c272c8446da47c12f8869fd32d (127.24.159.194:46507): tablet c52fb61129e04d3ab1b41b482ba9e22c (table test-table [id=aa102d571fda4bf59bbcf0668ea5c079]) successfully deleted
Sep 25 11:05:27 dist-test-slave-trms krb5kdc[12032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.193@KRBTEST.COM
I20260925 11:05:27.841019 12574 ts_tablet_manager.cc:1918] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:27.841180 12574 log.cc:1196] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/c52fb61129e04d3ab1b41b482ba9e22c
I20260925 11:05:27.841689 12574 ts_tablet_manager.cc:1939] T c52fb61129e04d3ab1b41b482ba9e22c P 96a44596d538406fb8357cfdc8c37221: Deleting consensus metadata
I20260925 11:05:27.843520 12061 catalog_manager.cc:4841] TS 96a44596d538406fb8357cfdc8c37221 (127.24.159.195:38137): tablet c52fb61129e04d3ab1b41b482ba9e22c (table test-table [id=aa102d571fda4bf59bbcf0668ea5c079]) successfully deleted
I20260925 11:05:27.844517 12575 ts_tablet_manager.cc:1918] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:27.844707 12575 log.cc:1196] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790334022505657-25215-0/minicluster-data/ts-0/wal/wals/c52fb61129e04d3ab1b41b482ba9e22c
I20260925 11:05:27.845237 12575 ts_tablet_manager.cc:1939] T c52fb61129e04d3ab1b41b482ba9e22c P 365b58ae62c84922852df34b37ac7d52: Deleting consensus metadata
I20260925 11:05:27.846905 12064 catalog_manager.cc:4841] TS 365b58ae62c84922852df34b37ac7d52 (127.24.159.193:34333): tablet c52fb61129e04d3ab1b41b482ba9e22c (table test-table [id=aa102d571fda4bf59bbcf0668ea5c079]) successfully deleted
I20260925 11:05:27.849267 12550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.816585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60754 (local address 127.24.159.193:34333)
0925 11:05:27.816769 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.816776 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.823494 (+ 6718us) server_negotiation.cc:407] Connection header received
0925 11:05:27.824360 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.824366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.824454 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.824538 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:27.826879 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.828012 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.829908 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.830632 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.830731 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.841781 (+ 11050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:27.841802 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:27.841823 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:27.841875 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:27.846242 (+ 4367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.847136 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.847147 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.847151 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.847348 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.847818 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.847822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.847824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.848208 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:27.848440 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.848804 (+ 364us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.848983 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:05:27.849965 12261 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:27.947032 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 12138
I20260925 11:05:27.963689 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 12272
Sep 25 11:05:27 dist-test-slave-trms krb5kdc[12032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.194@KRBTEST.COM
I20260925 11:05:27.977849 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:27.964367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52632 (local address 127.24.159.194:46507)
0925 11:05:27.964535 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:27.964541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:27.964561 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:05:27.964750 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:27.964756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:27.964827 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:27.964911 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:27.966274 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.967005 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.968242 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:27.968620 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:27.968762 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:27.972767 (+ 4005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:27.972780 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:27.972795 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:27.972833 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:27.975544 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.976110 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.976117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.976120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.976283 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:27.976584 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:27.976587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:27.976589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:27.976961 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:27.977220 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:27.977389 (+ 169us) server_negotiation.cc:299] Negotiation successful
0925 11:05:27.977583 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 11:05:27.978714 12395 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:28.071360 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 12272
I20260925 11:05:28.085742 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 12405
Sep 25 11:05:28 dist-test-slave-trms krb5kdc[12032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.195@KRBTEST.COM
I20260925 11:05:28.100246 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:28.086479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40558 (local address 127.24.159.195:38137)
0925 11:05:28.086670 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:28.086677 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:28.086697 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:05:28.086867 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:28.086873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:28.086936 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:28.087020 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:28.088352 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:28.089115 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:28.090421 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:28.090823 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:28.090934 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:28.095024 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:28.095040 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:28.095057 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:28.095097 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:28.097592 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:28.098264 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:28.098271 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:28.098273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:28.098415 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:28.098780 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:28.098784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:28.098786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:28.099123 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:28.099396 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:28.099729 (+ 333us) server_negotiation.cc:299] Negotiation successful
0925 11:05:28.099946 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 11:05:28.100878 12528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:28.189210 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 12405
I20260925 11:05:28.203404 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 12045
I20260925 11:05:28.204510 12107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:28.312373 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 12045
2026-09-25T11:05:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4029 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-25T11:05:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:28Z Disabled control of system clock
I20260925 11:05:28.383224 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37885
--webserver_interface=127.24.159.254
--webserver_port=0
--builtin_ntp_servers=127.24.159.212:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:37885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 11:05:28.702230 12585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:28.702508 12585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:28.702625 12585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:28.709697 12585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:28.709794 12585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:28.709857 12585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:28.709933 12585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:28.709982 12585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:28.719103 12585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:37885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:37885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:28.720399 12585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:28.722221 12585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:28.733383 12590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:28.733462 12591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:28.733492 12593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:28.734597 12585 server_base.cc:1034] running on GCE node
I20260925 11:05:28.735343 12585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:28.736681 12585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:28.737892 12585 hybrid_clock.cc:648] HybridClock initialized: now 1790334328737837 us; error 46 us; skew 500 ppm
I20260925 11:05:28.738279 12585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:28.740437 12585 webserver.cc:460] Webserver started at http://127.24.159.254:45593/ using document root <none> and password file <none>
I20260925 11:05:28.741123 12585 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:28.741221 12585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:28.741514 12585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:28.744449 12585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "4944442c7be74eac87c1f3130dab2b6e"
format_stamp: "Formatted at 2026-09-25 11:05:28 on dist-test-slave-trms"
server_key: "eb40d905fe56bb8904dece881221795d"
server_key_iv: "09f4606685e55de36afd215ba0132143"
server_key_version: "encryptionkey@0"
I20260925 11:05:28.745230 12585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "4944442c7be74eac87c1f3130dab2b6e"
format_stamp: "Formatted at 2026-09-25 11:05:28 on dist-test-slave-trms"
server_key: "eb40d905fe56bb8904dece881221795d"
server_key_iv: "09f4606685e55de36afd215ba0132143"
server_key_version: "encryptionkey@0"
I20260925 11:05:28.750269 12585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260925 11:05:28.753448 12599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:28.754734 12585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:05:28.754889 12585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "4944442c7be74eac87c1f3130dab2b6e"
format_stamp: "Formatted at 2026-09-25 11:05:28 on dist-test-slave-trms"
server_key: "eb40d905fe56bb8904dece881221795d"
server_key_iv: "09f4606685e55de36afd215ba0132143"
server_key_version: "encryptionkey@0"
I20260925 11:05:28.755075 12585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:28.767750 12585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:28.768493 12585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:28.768752 12585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:28.785262 12585 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:37885
I20260925 11:05:28.785269 12650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:37885 every 8 connection(s)
I20260925 11:05:28.786594 12585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:28.788203 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 12585
I20260925 11:05:28.788370 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:28.788816 25215 external_mini_cluster.cc:1428] Setting key c16af32fd47c91a32ef4e4a2380b5377
I20260925 11:05:28.790690 12651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:28.801471 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:28.790663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.24.159.254:37885)
0925 11:05:28.791666 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:28.791678 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:28.791719 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:05:28.793293 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:28.793314 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:28.793774 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:28.794154 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:28.794722 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:28.796082 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:28.797381 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:28.798037 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:28.798244 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:28.799304 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:28.799312 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:28.799329 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:28.799884 (+ 555us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:28.799954 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:28.799974 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:28.800086 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:28.800292 (+ 206us) server_negotiation.cc:299] Negotiation successful
0925 11:05:28.800580 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":369,"threads_started":1}
I20260925 11:05:28.801692 12651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e: Bootstrap starting.
I20260925 11:05:28.805001 12651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:28.805892 12651 log.cc:824] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:28.808405 12651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e: No bootstrap required, opened a new log
I20260925 11:05:28.816030 12651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4944442c7be74eac87c1f3130dab2b6e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37885 } }
I20260925 11:05:28.816371 12651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:28.816442 12651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4944442c7be74eac87c1f3130dab2b6e, State: Initialized, Role: FOLLOWER
I20260925 11:05:28.816926 12651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4944442c7be74eac87c1f3130dab2b6e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37885 } }
I20260925 11:05:28.817123 12651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:28.817214 12651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:28.817330 12651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:28.819371 12651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4944442c7be74eac87c1f3130dab2b6e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37885 } }
I20260925 11:05:28.819799 12651 leader_election.cc:302] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4944442c7be74eac87c1f3130dab2b6e; no voters:
I20260925 11:05:28.820358 12651 leader_election.cc:288] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:28.820497 12656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:28.821274 12656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [term 1 LEADER]: Becoming Leader. State: Replica: 4944442c7be74eac87c1f3130dab2b6e, State: Running, Role: LEADER
I20260925 11:05:28.821807 12656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4944442c7be74eac87c1f3130dab2b6e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37885 } }
I20260925 11:05:28.822585 12651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:28.824463 12657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4944442c7be74eac87c1f3130dab2b6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4944442c7be74eac87c1f3130dab2b6e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37885 } } }
I20260925 11:05:28.824889 12657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:28.825388 12658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4944442c7be74eac87c1f3130dab2b6e. Latest consensus state: current_term: 1 leader_uuid: "4944442c7be74eac87c1f3130dab2b6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4944442c7be74eac87c1f3130dab2b6e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 37885 } } }
I20260925 11:05:28.825699 12658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:28.827195 12665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:28.831925 12665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:28.839982 12665 catalog_manager.cc:1347] Generated new cluster ID: 48b399efc0e8437d82b5c4cb263a9bf4
I20260925 11:05:28.840091 12665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:28.857407 12665 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:28.859297 12665 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:28.869542 12665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e: Generated new TSK 0
I20260925 11:05:28.870600 12665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:28.874738 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.193:0
--local_ip_for_outbound_sockets=127.24.159.193
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37885
--builtin_ntp_servers=127.24.159.212:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260925 11:05:29.186784 12675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:29.187057 12675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:29.187187 12675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:29.194237 12675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:29.194344 12675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:29.194486 12675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.193
I20260925 11:05:29.203402 12675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.159.193
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.193
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:29.204663 12675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:29.206475 12675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:29.218644 12681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:29.218705 12680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:29.218683 12683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:29.219731 12675 server_base.cc:1034] running on GCE node
I20260925 11:05:29.220327 12675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:29.221254 12675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:29.222450 12675 hybrid_clock.cc:648] HybridClock initialized: now 1790334329222387 us; error 51 us; skew 500 ppm
I20260925 11:05:29.222765 12675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:29.224888 12675 webserver.cc:460] Webserver started at http://127.24.159.193:44059/ using document root <none> and password file <none>
I20260925 11:05:29.225535 12675 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:29.225632 12675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:29.225925 12675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:29.228704 12675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/data/instance:
uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d"
format_stamp: "Formatted at 2026-09-25 11:05:29 on dist-test-slave-trms"
server_key: "fd1c73bc18697997741afdf35e404c32"
server_key_iv: "a57178f708d2600de0f2e741cbf2b040"
server_key_version: "encryptionkey@0"
I20260925 11:05:29.229446 12675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance:
uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d"
format_stamp: "Formatted at 2026-09-25 11:05:29 on dist-test-slave-trms"
server_key: "fd1c73bc18697997741afdf35e404c32"
server_key_iv: "a57178f708d2600de0f2e741cbf2b040"
server_key_version: "encryptionkey@0"
I20260925 11:05:29.234180 12675 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 11:05:29.237512 12689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:29.238875 12675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:05:29.239027 12675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/wal
uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d"
format_stamp: "Formatted at 2026-09-25 11:05:29 on dist-test-slave-trms"
server_key: "fd1c73bc18697997741afdf35e404c32"
server_key_iv: "a57178f708d2600de0f2e741cbf2b040"
server_key_version: "encryptionkey@0"
I20260925 11:05:29.239221 12675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:29.258801 12675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:29.259610 12675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:29.259884 12675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:29.260792 12675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:29.262233 12675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:29.262331 12675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:29.262432 12675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:29.262502 12675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:29.291543 12675 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.193:43085
I20260925 11:05:29.291571 12801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.193:43085 every 8 connection(s)
I20260925 11:05:29.292809 12675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/data/info.pb
I20260925 11:05:29.297253 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 12675
I20260925 11:05:29.297493 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-0/wal/instance
I20260925 11:05:29.298050 25215 external_mini_cluster.cc:1428] Setting key d7365996324353bd5e30d7d9746a6618
I20260925 11:05:29.300974 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.194:0
--local_ip_for_outbound_sockets=127.24.159.194
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37885
--builtin_ntp_servers=127.24.159.212:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 11:05:29.306258 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:29.295256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.193:53573 (local address 127.24.159.254:37885)
0925 11:05:29.295425 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:29.295430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:29.296691 (+ 1261us) server_negotiation.cc:407] Connection header received
0925 11:05:29.299048 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:29.299052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:29.299134 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:29.299232 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:29.300499 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:29.301418 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:29.303582 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:29.303993 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:29.304089 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:29.305566 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:29.305570 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:29.305573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:29.305592 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:29.305621 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:29.305644 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:29.305707 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:29.305927 (+ 220us) server_negotiation.cc:299] Negotiation successful
0925 11:05:29.306004 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:05:29.307659 12804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:29.295687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37885 (local address 127.24.159.193:53573)
0925 11:05:29.296558 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:29.296593 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:29.298754 (+ 2161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:29.299446 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:29.299454 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:29.299546 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:29.300290 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:29.300328 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:29.301581 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:29.301586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:29.303448 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:29.303464 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:29.304206 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:29.304212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:29.304490 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:29.305196 (+ 706us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:05:29.305199 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:29.305257 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:05:29.305370 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:29.305723 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:29.305733 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:29.305740 (+ 7us) client_negotiation.cc:769] Sending connection context
0925 11:05:29.305916 (+ 176us) client_negotiation.cc:240] Negotiation successful
0925 11:05:29.306353 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":249,"threads_started":1}
I20260925 11:05:29.309156 12802 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37885
I20260925 11:05:29.309470 12802 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:29.310319 12802 heartbeater.cc:511] Master 127.24.159.254:37885 requested a full tablet report, sending...
I20260925 11:05:29.312474 12616 ts_manager.cc:194] Registered new tserver with Master: 97e6d1a59efc444fbc6a6028e4d1bb2d (127.24.159.193:43085)
I20260925 11:05:29.314440 12616 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.193:53573
W20260925 11:05:29.624096 12805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:29.624369 12805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:29.624462 12805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:29.631700 12805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:29.631800 12805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:29.631912 12805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.194
I20260925 11:05:29.640901 12805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.159.194
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.194
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:29.642107 12805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:29.643965 12805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:29.656584 12810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:29.656523 12813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:29.656508 12811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:29.657320 12805 server_base.cc:1034] running on GCE node
I20260925 11:05:29.657897 12805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:29.658869 12805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:29.660094 12805 hybrid_clock.cc:648] HybridClock initialized: now 1790334329660030 us; error 51 us; skew 500 ppm
I20260925 11:05:29.660530 12805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:29.662755 12805 webserver.cc:460] Webserver started at http://127.24.159.194:44781/ using document root <none> and password file <none>
I20260925 11:05:29.663426 12805 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:29.663527 12805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:29.663826 12805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:29.666690 12805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/data/instance:
uuid: "76e0eb8aceb94db6b24a754176e6f933"
format_stamp: "Formatted at 2026-09-25 11:05:29 on dist-test-slave-trms"
server_key: "50159af4bc532754612e389893841134"
server_key_iv: "9d350c9c0c15ede03ae09341dfabe722"
server_key_version: "encryptionkey@0"
I20260925 11:05:29.667462 12805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance:
uuid: "76e0eb8aceb94db6b24a754176e6f933"
format_stamp: "Formatted at 2026-09-25 11:05:29 on dist-test-slave-trms"
server_key: "50159af4bc532754612e389893841134"
server_key_iv: "9d350c9c0c15ede03ae09341dfabe722"
server_key_version: "encryptionkey@0"
I20260925 11:05:29.672649 12805 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260925 11:05:29.676115 12819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:29.677429 12805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:05:29.677587 12805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/wal
uuid: "76e0eb8aceb94db6b24a754176e6f933"
format_stamp: "Formatted at 2026-09-25 11:05:29 on dist-test-slave-trms"
server_key: "50159af4bc532754612e389893841134"
server_key_iv: "9d350c9c0c15ede03ae09341dfabe722"
server_key_version: "encryptionkey@0"
I20260925 11:05:29.677788 12805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:29.700776 12805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:29.701575 12805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:29.701850 12805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:29.702823 12805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:29.704257 12805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:29.704350 12805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:29.704455 12805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:29.704531 12805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:29.734498 12805 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.194:41797
I20260925 11:05:29.734565 12931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.194:41797 every 8 connection(s)
I20260925 11:05:29.735761 12805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/data/info.pb
I20260925 11:05:29.737540 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 12805
I20260925 11:05:29.737707 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/wal/instance
I20260925 11:05:29.738245 25215 external_mini_cluster.cc:1428] Setting key 7a3fb0de96790d7e4b0412b2b9ae3b1e
I20260925 11:05:29.741297 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.159.195:0
--local_ip_for_outbound_sockets=127.24.159.195
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37885
--builtin_ntp_servers=127.24.159.212:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260925 11:05:29.748730 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:29.738318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:53021 (local address 127.24.159.254:37885)
0925 11:05:29.738495 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:29.738500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:29.740112 (+ 1612us) server_negotiation.cc:407] Connection header received
0925 11:05:29.742148 (+ 2036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:29.742154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:29.742257 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:29.742360 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:29.743532 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:29.744444 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:29.745993 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:29.746629 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:29.746749 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:29.748021 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:29.748026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:29.748030 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:29.748050 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:29.748087 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:29.748100 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:29.748191 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:29.748352 (+ 161us) server_negotiation.cc:299] Negotiation successful
0925 11:05:29.748448 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260925 11:05:29.749951 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:29.739047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37885 (local address 127.24.159.194:53021)
0925 11:05:29.739954 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:29.739997 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:29.741862 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:29.742589 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:29.742598 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:29.742704 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:29.743362 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:29.743389 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:29.744598 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:29.744601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:29.745846 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:29.745860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:29.746823 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:29.746828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:29.747038 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:29.747693 (+ 655us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:05:29.747695 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:29.747735 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:05:29.747829 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:29.748186 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:29.748191 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:29.748196 (+ 5us) client_negotiation.cc:769] Sending connection context
0925 11:05:29.748328 (+ 132us) client_negotiation.cc:240] Negotiation successful
0925 11:05:29.748661 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":277,"threads_started":1}
I20260925 11:05:29.751499 12932 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37885
I20260925 11:05:29.751797 12932 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:29.752532 12932 heartbeater.cc:511] Master 127.24.159.254:37885 requested a full tablet report, sending...
I20260925 11:05:29.753923 12616 ts_manager.cc:194] Registered new tserver with Master: 76e0eb8aceb94db6b24a754176e6f933 (127.24.159.194:41797)
I20260925 11:05:29.754756 12616 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.194:53021
W20260925 11:05:30.064128 12935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:30.064411 12935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:30.064532 12935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:30.071656 12935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:30.071751 12935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:30.071880 12935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.159.195
I20260925 11:05:30.080689 12935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.159.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.159.195
--webserver_port=0
--tserver_master_addrs=127.24.159.254:37885
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.159.195
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:30.081959 12935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:30.083717 12935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:30.095956 12941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:30.096017 12940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:30.095928 12943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:30.097007 12935 server_base.cc:1034] running on GCE node
I20260925 11:05:30.097580 12935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:30.098552 12935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:30.099788 12935 hybrid_clock.cc:648] HybridClock initialized: now 1790334330099742 us; error 46 us; skew 500 ppm
I20260925 11:05:30.100108 12935 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260925 11:05:30.102370 12935 webserver.cc:460] Webserver started at http://127.24.159.195:34983/ using document root <none> and password file <none>
I20260925 11:05:30.103042 12935 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:30.103145 12935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:30.103430 12935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:30.106258 12935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/data/instance:
uuid: "17d3d46aff4640489b5fc6764d29f866"
format_stamp: "Formatted at 2026-09-25 11:05:30 on dist-test-slave-trms"
server_key: "7c44b11159200db81c9e586b6404dc1e"
server_key_iv: "9ec40e6fbd3e4eb3a5cff5a8d1553d09"
server_key_version: "encryptionkey@0"
I20260925 11:05:30.107013 12935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance:
uuid: "17d3d46aff4640489b5fc6764d29f866"
format_stamp: "Formatted at 2026-09-25 11:05:30 on dist-test-slave-trms"
server_key: "7c44b11159200db81c9e586b6404dc1e"
server_key_iv: "9ec40e6fbd3e4eb3a5cff5a8d1553d09"
server_key_version: "encryptionkey@0"
I20260925 11:05:30.111770 12935 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260925 11:05:30.115023 12949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:30.116291 12935 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:05:30.116441 12935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/wal
uuid: "17d3d46aff4640489b5fc6764d29f866"
format_stamp: "Formatted at 2026-09-25 11:05:30 on dist-test-slave-trms"
server_key: "7c44b11159200db81c9e586b6404dc1e"
server_key_iv: "9ec40e6fbd3e4eb3a5cff5a8d1553d09"
server_key_version: "encryptionkey@0"
I20260925 11:05:30.116639 12935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:30.140190 12935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:30.141044 12935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:30.141328 12935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:30.142367 12935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260925 11:05:30.143880 12935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260925 11:05:30.143973 12935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:30.144073 12935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260925 11:05:30.144140 12935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:30.173130 12935 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.195:35611
I20260925 11:05:30.173207 13061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.195:35611 every 8 connection(s)
I20260925 11:05:30.174480 12935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/data/info.pb
I20260925 11:05:30.176673 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 12935
I20260925 11:05:30.176818 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/wal/instance
I20260925 11:05:30.177313 25215 external_mini_cluster.cc:1428] Setting key 566e9b3b730a279236b472414e2ef634
I20260925 11:05:30.186839 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.177069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.195:35979 (local address 127.24.159.254:37885)
0925 11:05:30.177279 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.177286 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.178871 (+ 1585us) server_negotiation.cc:407] Connection header received
0925 11:05:30.180908 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.180912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.180980 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.181062 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:30.182129 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.182866 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.184430 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.184846 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.184938 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.186225 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:30.186230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:30.186235 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:30.186253 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:30.186281 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:30.186293 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:30.186358 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.186532 (+ 174us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.186606 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260925 11:05:30.187866 13064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.177605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:37885 (local address 127.24.159.195:35979)
0925 11:05:30.178718 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.178766 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:30.180626 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:30.181248 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:30.181257 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:30.181352 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:05:30.181979 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.181999 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.183024 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.183028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.184294 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.184306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.185056 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.185061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.185267 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.185926 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0925 11:05:30.185929 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:05:30.185969 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0925 11:05:30.186050 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:05:30.186366 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:05:30.186373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:05:30.186379 (+ 6us) client_negotiation.cc:769] Sending connection context
0925 11:05:30.186520 (+ 141us) client_negotiation.cc:240] Negotiation successful
0925 11:05:30.186825 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":333,"threads_started":1}
I20260925 11:05:30.189121 13062 heartbeater.cc:347] Connected to a master server at 127.24.159.254:37885
I20260925 11:05:30.189416 13062 heartbeater.cc:464] Registering TS with master...
I20260925 11:05:30.190112 13062 heartbeater.cc:511] Master 127.24.159.254:37885 requested a full tablet report, sending...
I20260925 11:05:30.191452 12616 ts_manager.cc:194] Registered new tserver with Master: 17d3d46aff4640489b5fc6764d29f866 (127.24.159.195:35611)
I20260925 11:05:30.192384 25215 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260925 11:05:30.192459 12616 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.159.195:35979
I20260925 11:05:30.202368 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.196768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52022 (local address 127.24.159.254:37885)
0925 11:05:30.196924 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.196928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.197371 (+ 443us) server_negotiation.cc:407] Connection header received
0925 11:05:30.197522 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.197527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.197627 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.197710 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:30.198184 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.198927 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.200228 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.200636 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.200742 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.201683 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:30.201689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:30.201693 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:30.201710 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:30.201739 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:30.201750 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:30.201821 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.201943 (+ 122us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.202029 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:05:30.205989 12616 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52022:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260925 11:05:30.212656 12616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260925 11:05:30.235272 13076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.224909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43085 (local address 127.0.0.1:57538)
0925 11:05:30.226699 (+ 1790us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.226716 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:30.226836 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:30.227500 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:30.227507 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:30.227535 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:30.228034 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.228046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.230048 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.230053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.233023 (+ 2970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.233034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.234287 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.234294 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.234496 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.234549 (+ 53us) client_negotiation.cc:769] Sending connection context
0925 11:05:30.234670 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 11:05:30.234852 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1676,"mutex_wait_us":171,"thread_start_us":1514,"threads_started":1}
I20260925 11:05:30.235352 13075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.224832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48056 (local address 127.24.159.194:41797)
0925 11:05:30.226882 (+ 2050us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.226891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.226920 (+ 29us) server_negotiation.cc:407] Connection header received
0925 11:05:30.227016 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.227023 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.227272 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.227972 (+ 700us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:30.228605 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.230043 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.233031 (+ 2988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.234345 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.234638 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.234847 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.234912 (+ 65us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.235036 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1840,"thread_start_us":1713,"threads_started":1}
I20260925 11:05:30.235646 13074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.224498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.194:41797 (local address 127.0.0.1:48056)
0925 11:05:30.225572 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.225612 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:30.225782 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:30.227543 (+ 1761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:30.227548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:30.227577 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:30.228362 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.228375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.230179 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.230184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.232810 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.232834 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.234509 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.234516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.234731 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.234761 (+ 30us) client_negotiation.cc:769] Sending connection context
0925 11:05:30.234834 (+ 73us) client_negotiation.cc:240] Negotiation successful
0925 11:05:30.234930 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":244,"thread_start_us":682,"threads_started":1}
I20260925 11:05:30.236274 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.225403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57538 (local address 127.24.159.193:43085)
0925 11:05:30.226390 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.226400 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.226823 (+ 423us) server_negotiation.cc:407] Connection header received
0925 11:05:30.226976 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.226985 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.227264 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.228541 (+ 1277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:30.228764 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.229937 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.233187 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.234096 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.234264 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.234493 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.235761 (+ 1268us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.235937 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":649,"threads_started":1}
I20260925 11:05:30.236316 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.226776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:35611 (local address 127.0.0.1:47372)
0925 11:05:30.227218 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.227234 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:30.227386 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:30.229158 (+ 1772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:30.229163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:30.229189 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:30.229487 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.229496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.230956 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.230961 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.232735 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.232747 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.234509 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.234516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.235748 (+ 1232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.235780 (+ 32us) client_negotiation.cc:769] Sending connection context
0925 11:05:30.235874 (+ 94us) client_negotiation.cc:240] Negotiation successful
0925 11:05:30.235993 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":612,"thread_start_us":690,"threads_started":1}
I20260925 11:05:30.239447 12737 tablet_service.cc:1467] Processing CreateTablet for tablet fd99ab3bd76d4e169bb752be22a2d63e (DEFAULT_TABLE table=test-table [id=280c60f4eb354fcf89788e8e73be6a38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:30.239454 12867 tablet_service.cc:1467] Processing CreateTablet for tablet fd99ab3bd76d4e169bb752be22a2d63e (DEFAULT_TABLE table=test-table [id=280c60f4eb354fcf89788e8e73be6a38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:30.240633 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.227289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.24.159.195:35611)
0925 11:05:30.228619 (+ 1330us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.228626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.228666 (+ 40us) server_negotiation.cc:407] Connection header received
0925 11:05:30.228760 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.228767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.228968 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.229160 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:30.229656 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.230823 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.232891 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.234285 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.234674 (+ 389us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.234847 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.240100 (+ 5253us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.240273 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1191,"thread_start_us":1059,"threads_started":1}
I20260925 11:05:30.240747 12867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd99ab3bd76d4e169bb752be22a2d63e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:30.240751 12737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd99ab3bd76d4e169bb752be22a2d63e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:30.244366 12997 tablet_service.cc:1467] Processing CreateTablet for tablet fd99ab3bd76d4e169bb752be22a2d63e (DEFAULT_TABLE table=test-table [id=280c60f4eb354fcf89788e8e73be6a38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260925 11:05:30.245791 12997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd99ab3bd76d4e169bb752be22a2d63e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:30.251875 13080 tablet_bootstrap.cc:492] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: Bootstrap starting.
I20260925 11:05:30.254992 13080 tablet_bootstrap.cc:654] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:30.255636 13081 tablet_bootstrap.cc:492] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d: Bootstrap starting.
I20260925 11:05:30.256323 13080 log.cc:824] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:30.258824 13083 tablet_bootstrap.cc:492] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: Bootstrap starting.
I20260925 11:05:30.259070 13081 tablet_bootstrap.cc:654] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:30.260069 13080 tablet_bootstrap.cc:492] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: No bootstrap required, opened a new log
I20260925 11:05:30.260265 13081 log.cc:824] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:30.260350 13080 ts_tablet_manager.cc:1397] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260925 11:05:30.261773 13083 tablet_bootstrap.cc:654] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:30.262980 13083 log.cc:824] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:30.263372 13081 tablet_bootstrap.cc:492] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d: No bootstrap required, opened a new log
I20260925 11:05:30.263628 13081 ts_tablet_manager.cc:1397] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260925 11:05:30.265352 13083 tablet_bootstrap.cc:492] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: No bootstrap required, opened a new log
I20260925 11:05:30.265569 13083 ts_tablet_manager.cc:1397] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260925 11:05:30.270555 13080 raft_consensus.cc:348] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } }
I20260925 11:05:30.271015 13080 raft_consensus.cc:374] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:30.271148 13080 raft_consensus.cc:729] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76e0eb8aceb94db6b24a754176e6f933, State: Initialized, Role: FOLLOWER
I20260925 11:05:30.271878 13080 consensus_queue.cc:260] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } }
I20260925 11:05:30.272359 13083 raft_consensus.cc:348] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } }
I20260925 11:05:30.272830 13083 raft_consensus.cc:374] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:30.272962 13083 raft_consensus.cc:729] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17d3d46aff4640489b5fc6764d29f866, State: Initialized, Role: FOLLOWER
I20260925 11:05:30.273696 13083 consensus_queue.cc:260] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } }
I20260925 11:05:30.273743 13081 raft_consensus.cc:348] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } }
I20260925 11:05:30.274303 13081 raft_consensus.cc:374] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:30.274438 13081 raft_consensus.cc:729] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97e6d1a59efc444fbc6a6028e4d1bb2d, State: Initialized, Role: FOLLOWER
I20260925 11:05:30.274909 12932 heartbeater.cc:503] Master 127.24.159.254:37885 was elected leader, sending a full tablet report...
I20260925 11:05:30.275125 13081 consensus_queue.cc:260] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } }
I20260925 11:05:30.275703 13080 ts_tablet_manager.cc:1428] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: Time spent starting tablet: real 0.015s user 0.012s sys 0.003s
I20260925 11:05:30.277197 12802 heartbeater.cc:503] Master 127.24.159.254:37885 was elected leader, sending a full tablet report...
I20260925 11:05:30.277245 13083 ts_tablet_manager.cc:1428] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260925 11:05:30.277734 13081 ts_tablet_manager.cc:1428] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260925 11:05:30.278467 13062 heartbeater.cc:503] Master 127.24.159.254:37885 was elected leader, sending a full tablet report...
W20260925 11:05:30.294812 12803 tablet.cc:2369] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:05:30.333832 13086 raft_consensus.cc:482] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260925 11:05:30.334146 13086 raft_consensus.cc:504] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } }
I20260925 11:05:30.335824 13086 leader_election.cc:288] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97e6d1a59efc444fbc6a6028e4d1bb2d (127.24.159.193:43085), 17d3d46aff4640489b5fc6764d29f866 (127.24.159.195:35611)
I20260925 11:05:30.342056 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.336182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:37197 (local address 127.24.159.193:43085)
0925 11:05:30.336399 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.336406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.336896 (+ 490us) server_negotiation.cc:407] Connection header received
0925 11:05:30.337128 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.337135 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.337229 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.337328 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:30.337906 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.338629 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.340490 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.341220 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.341363 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.341432 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.341690 (+ 258us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.341811 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:05:30.342257 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.337103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.159.194:35285 (local address 127.24.159.195:35611)
0925 11:05:30.337304 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.337310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.337533 (+ 223us) server_negotiation.cc:407] Connection header received
0925 11:05:30.337707 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.337713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.337770 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.337839 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0925 11:05:30.338322 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.338989 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.340667 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.341420 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.341577 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.341640 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.341889 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.342005 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 11:05:30.342232 13089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.336181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43085 (local address 127.24.159.194:37197)
0925 11:05:30.336773 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.336794 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:30.336951 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:30.337386 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:30.337391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:30.337418 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:30.337765 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.337774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.338756 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.338760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.340314 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.340327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.341400 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.341406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.341545 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.341571 (+ 26us) client_negotiation.cc:769] Sending connection context
0925 11:05:30.341649 (+ 78us) client_negotiation.cc:240] Negotiation successful
0925 11:05:30.341778 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":75,"spinlock_wait_cycles":12160,"thread_start_us":317,"threads_started":1}
I20260925 11:05:30.342526 13090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.336889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.195:35611 (local address 127.24.159.194:35285)
0925 11:05:30.337441 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.337453 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0925 11:05:30.337583 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:05:30.337869 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:05:30.337873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:05:30.337894 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0925 11:05:30.338169 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.338177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.339117 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.339123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.340497 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:05:30.340509 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.341588 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.341594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:05:30.341754 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.341776 (+ 22us) client_negotiation.cc:769] Sending connection context
0925 11:05:30.341848 (+ 72us) client_negotiation.cc:240] Negotiation successful
0925 11:05:30.341963 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":279,"threads_started":1}
I20260925 11:05:30.343334 12757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd99ab3bd76d4e169bb752be22a2d63e" candidate_uuid: "76e0eb8aceb94db6b24a754176e6f933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" is_pre_election: true
I20260925 11:05:30.343382 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd99ab3bd76d4e169bb752be22a2d63e" candidate_uuid: "76e0eb8aceb94db6b24a754176e6f933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17d3d46aff4640489b5fc6764d29f866" is_pre_election: true
I20260925 11:05:30.343868 12757 raft_consensus.cc:2454] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76e0eb8aceb94db6b24a754176e6f933 in term 0.
I20260925 11:05:30.343894 13017 raft_consensus.cc:2454] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76e0eb8aceb94db6b24a754176e6f933 in term 0.
I20260925 11:05:30.344484 12822 leader_election.cc:302] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17d3d46aff4640489b5fc6764d29f866, 76e0eb8aceb94db6b24a754176e6f933; no voters:
I20260925 11:05:30.344821 13086 raft_consensus.cc:2781] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260925 11:05:30.344944 13086 raft_consensus.cc:482] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260925 11:05:30.345041 13086 raft_consensus.cc:3037] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:30.347397 13086 raft_consensus.cc:504] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } }
I20260925 11:05:30.347927 13086 leader_election.cc:288] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [CANDIDATE]: Term 1 election: Requested vote from peers 97e6d1a59efc444fbc6a6028e4d1bb2d (127.24.159.193:43085), 17d3d46aff4640489b5fc6764d29f866 (127.24.159.195:35611)
I20260925 11:05:30.348337 12757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd99ab3bd76d4e169bb752be22a2d63e" candidate_uuid: "76e0eb8aceb94db6b24a754176e6f933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d"
I20260925 11:05:30.348426 13017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd99ab3bd76d4e169bb752be22a2d63e" candidate_uuid: "76e0eb8aceb94db6b24a754176e6f933" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17d3d46aff4640489b5fc6764d29f866"
I20260925 11:05:30.348502 12757 raft_consensus.cc:3037] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:30.348621 13017 raft_consensus.cc:3037] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:30.350771 12757 raft_consensus.cc:2454] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76e0eb8aceb94db6b24a754176e6f933 in term 1.
I20260925 11:05:30.350944 13017 raft_consensus.cc:2454] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76e0eb8aceb94db6b24a754176e6f933 in term 1.
I20260925 11:05:30.351114 12822 leader_election.cc:302] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76e0eb8aceb94db6b24a754176e6f933, 97e6d1a59efc444fbc6a6028e4d1bb2d; no voters:
I20260925 11:05:30.351377 13086 raft_consensus.cc:2781] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:30.351765 13086 raft_consensus.cc:686] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 1 LEADER]: Becoming Leader. State: Replica: 76e0eb8aceb94db6b24a754176e6f933, State: Running, Role: LEADER
I20260925 11:05:30.352273 13086 consensus_queue.cc:237] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } }
I20260925 11:05:30.356703 12615 catalog_manager.cc:5495] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76e0eb8aceb94db6b24a754176e6f933 (127.24.159.194). New cstate: current_term: 1 leader_uuid: "76e0eb8aceb94db6b24a754176e6f933" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76e0eb8aceb94db6b24a754176e6f933" member_type: VOTER last_known_addr { host: "127.24.159.194" port: 41797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 } health_report { overall_health: UNKNOWN } } }
I20260925 11:05:30.387393 13075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.382308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48072 (local address 127.24.159.194:41797)
0925 11:05:30.382511 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.382517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.382537 (+ 20us) server_negotiation.cc:407] Connection header received
0925 11:05:30.382670 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.382675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.382754 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.382890 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:05:30.383254 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.383962 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.385329 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.385732 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.385849 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.386344 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:05:30.386847 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:05:30.386977 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.387080 (+ 103us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.387154 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 11:05:30.395337 13017 raft_consensus.cc:1261] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [term 1 FOLLOWER]: Refusing update from remote peer 76e0eb8aceb94db6b24a754176e6f933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:30.395358 12757 raft_consensus.cc:1261] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [term 1 FOLLOWER]: Refusing update from remote peer 76e0eb8aceb94db6b24a754176e6f933: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:05:30.396234 13091 consensus_queue.cc:1035] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97e6d1a59efc444fbc6a6028e4d1bb2d" member_type: VOTER last_known_addr { host: "127.24.159.193" port: 43085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:30.396535 13086 consensus_queue.cc:1035] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17d3d46aff4640489b5fc6764d29f866" member_type: VOTER last_known_addr { host: "127.24.159.195" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:05:30.407222 13094 mvcc.cc:205] Tried to move back new op lower bound from 7333209417289605120 to 7333209417127022592. Current Snapshot: MvccSnapshot[applied={T|T < 7333209417289605120}]
I20260925 11:05:30.420226 12616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:30.420569 12616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52022:
table { table_name: "test-table" } modify_external_catalogs: true
I20260925 11:05:30.424587 12616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4944442c7be74eac87c1f3130dab2b6e: Sending DeleteTablet for 3 replicas of tablet fd99ab3bd76d4e169bb752be22a2d63e
I20260925 11:05:30.425796 12867 tablet_service.cc:1514] Processing DeleteTablet for tablet fd99ab3bd76d4e169bb752be22a2d63e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:30 UTC) from {username='slave'} at 127.0.0.1:48056
I20260925 11:05:30.426928 12997 tablet_service.cc:1514] Processing DeleteTablet for tablet fd99ab3bd76d4e169bb752be22a2d63e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:30 UTC) from {username='slave'} at 127.0.0.1:47372
I20260925 11:05:30.427631 12737 tablet_service.cc:1514] Processing DeleteTablet for tablet fd99ab3bd76d4e169bb752be22a2d63e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-25 11:05:30 UTC) from {username='slave'} at 127.0.0.1:57538
I20260925 11:05:30.429152 13103 tablet_replica.cc:330] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: stopping tablet replica
I20260925 11:05:30.430485 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 12675
I20260925 11:05:30.438710 13103 raft_consensus.cc:2229] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 1 LEADER]: Raft consensus shutting down.
I20260925 11:05:30.439579 13103 raft_consensus.cc:2258] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:30.441268 13104 tablet_replica.cc:330] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: stopping tablet replica
I20260925 11:05:30.441855 13104 raft_consensus.cc:2229] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:30.441923 13103 ts_tablet_manager.cc:1905] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:30.442485 13104 raft_consensus.cc:2258] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:30.443526 13105 tablet_replica.cc:330] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d: stopping tablet replica
I20260925 11:05:30.444514 13104 ts_tablet_manager.cc:1905] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:30.448921 13105 raft_consensus.cc:2229] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260925 11:05:30.449070 13104 ts_tablet_manager.cc:1918] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:30.449249 13104 log.cc:1196] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-2/wal/wals/fd99ab3bd76d4e169bb752be22a2d63e
I20260925 11:05:30.449638 13105 raft_consensus.cc:2258] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260925 11:05:30.449726 13104 ts_tablet_manager.cc:1939] T fd99ab3bd76d4e169bb752be22a2d63e P 17d3d46aff4640489b5fc6764d29f866: Deleting consensus metadata
I20260925 11:05:30.450150 13103 ts_tablet_manager.cc:1918] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260925 11:05:30.450348 13103 log.cc:1196] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: Deleting WAL directory at /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790334022505657-25215-0/minicluster-data/ts-1/wal/wals/fd99ab3bd76d4e169bb752be22a2d63e
I20260925 11:05:30.451066 13103 ts_tablet_manager.cc:1939] T fd99ab3bd76d4e169bb752be22a2d63e P 76e0eb8aceb94db6b24a754176e6f933: Deleting consensus metadata
I20260925 11:05:30.451761 12602 catalog_manager.cc:4841] TS 17d3d46aff4640489b5fc6764d29f866 (127.24.159.195:35611): tablet fd99ab3bd76d4e169bb752be22a2d63e (table test-table [id=280c60f4eb354fcf89788e8e73be6a38]) successfully deleted
I20260925 11:05:30.452386 13077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.432031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57546 (local address 127.24.159.193:43085)
0925 11:05:30.432218 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.432224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.438690 (+ 6466us) server_negotiation.cc:407] Connection header received
0925 11:05:30.439015 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.439021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.439161 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.439241 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:30.439913 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.441038 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.442802 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.443498 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.443600 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.450700 (+ 7100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:30.450711 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:30.450728 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:30.451456 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:30.451518 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:30.451543 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:30.451631 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.451932 (+ 301us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.452073 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 11:05:30.453814 12797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:30.453979 12603 catalog_manager.cc:4841] TS 76e0eb8aceb94db6b24a754176e6f933 (127.24.159.194:41797): tablet fd99ab3bd76d4e169bb752be22a2d63e (table test-table [id=280c60f4eb354fcf89788e8e73be6a38]) successfully deleted
I20260925 11:05:30.453944 13105 ts_tablet_manager.cc:1905] T fd99ab3bd76d4e169bb752be22a2d63e P 97e6d1a59efc444fbc6a6028e4d1bb2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260925 11:05:30.543229 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 12675
W20260925 11:05:30.564031 12602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.159.193:43085 (error 108)
I20260925 11:05:30.564278 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 12805
I20260925 11:05:30.566105 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.565598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43085 (local address 127.0.0.1:57552)
0925 11:05:30.565793 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.565922 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260925 11:05:30.566393 12602 catalog_manager.cc:4584] TS 97e6d1a59efc444fbc6a6028e4d1bb2d (127.24.159.193:43085): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd99ab3bd76d4e169bb752be22a2d63e: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43085: connect: Connection refused (error 111)
I20260925 11:05:30.571266 13075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.565169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48088 (local address 127.24.159.194:41797)
0925 11:05:30.565354 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.565361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.565380 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:05:30.565531 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.565536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.565657 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.565758 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:30.566237 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.567059 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.568286 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.568710 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.568817 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.569825 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:30.569834 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:30.569851 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:30.570560 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:30.570617 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:30.570638 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:30.570733 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.570846 (+ 113us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.570946 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:05:30.571967 12927 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:30.619863 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.619374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43085 (local address 127.0.0.1:57556)
0925 11:05:30.619582 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.619671 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260925 11:05:30.620903 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.620542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43085 (local address 127.0.0.1:57566)
0925 11:05:30.620694 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.620760 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260925 11:05:30.681037 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 12805
I20260925 11:05:30.681530 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.681043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43085 (local address 127.0.0.1:57582)
0925 11:05:30.681251 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.681340 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260925 11:05:30.682668 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.682242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43085 (local address 127.0.0.1:57586)
0925 11:05:30.682424 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.682517 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260925 11:05:30.696488 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 12935
I20260925 11:05:30.702999 13079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.697191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47386 (local address 127.24.159.195:35611)
0925 11:05:30.697376 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:30.697382 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:30.697401 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:05:30.697584 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:30.697588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:30.697677 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:30.697763 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:30.698225 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.698957 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.700163 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:30.700555 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:30.700662 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:30.701636 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:30.701646 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:30.701662 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:30.702291 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:30.702357 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:30.702377 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:30.702474 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:30.702598 (+ 124us) server_negotiation.cc:299] Negotiation successful
0925 11:05:30.702701 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:05:30.703589 13057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:30.763823 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.763278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43085 (local address 127.0.0.1:57588)
0925 11:05:30.763509 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.763628 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260925 11:05:30.764928 13078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:30.764497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.193:43085 (local address 127.0.0.1:57602)
0925 11:05:30.764715 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 11:05:30.764800 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.193:43085: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260925 11:05:30.790719 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 12935
I20260925 11:05:30.808565 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 12585
I20260925 11:05:30.809623 12646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:30.916543 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 12585
2026-09-25T11:05:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2607 ms)
[----------] 6 tests from EncryptionPolicyTest (20070 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:31 dist-test-slave-trms krb5kdc[13114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:31 dist-test-slave-trms krb5kdc[13114](info): set up 2 sockets
Sep 25 11:05:31 dist-test-slave-trms krb5kdc[13114](info): commencing operation
krb5kdc: starting...
W20260925 11:05:31.934715 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:32 dist-test-slave-trms krb5kdc[13114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:32.175812 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44177
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:32.495635 13127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:32.495923 13127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:32.496022 13127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:32.503234 13127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:32.503345 13127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:32.503389 13127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:32.503451 13127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:32.503504 13127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:32.513187 13127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44177
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:32.514447 13127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:32.515705 13127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:32.527058 13132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:32.527138 13133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:32.527086 13135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:32.527570 13127 server_base.cc:1034] running on GCE node
I20260925 11:05:32.528481 13127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:32.529886 13127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:32.531101 13127 hybrid_clock.cc:648] HybridClock initialized: now 1790334332531059 us; error 42 us; skew 500 ppm
I20260925 11:05:32.531454 13127 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:32 dist-test-slave-trms krb5kdc[13114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334332, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:32.534361 13127 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:32.535808 13127 webserver.cc:460] Webserver started at http://127.0.0.1:42557/ using document root <none> and password file <none>
I20260925 11:05:32.536463 13127 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:32.536577 13127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:32.536854 13127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:32.539775 13127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "c728af672e0741fba2d29553ff198dd0"
format_stamp: "Formatted at 2026-09-25 11:05:32 on dist-test-slave-trms"
server_key: "40362531da3fe4603ec5074661f19c84"
server_key_iv: "6b710bc8dc4c1304c1d2603d03555150"
server_key_version: "encryptionkey@0"
I20260925 11:05:32.540532 13127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "c728af672e0741fba2d29553ff198dd0"
format_stamp: "Formatted at 2026-09-25 11:05:32 on dist-test-slave-trms"
server_key: "40362531da3fe4603ec5074661f19c84"
server_key_iv: "6b710bc8dc4c1304c1d2603d03555150"
server_key_version: "encryptionkey@0"
I20260925 11:05:32.545657 13127 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260925 11:05:32.549232 13142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:32.550635 13127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260925 11:05:32.550796 13127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "c728af672e0741fba2d29553ff198dd0"
format_stamp: "Formatted at 2026-09-25 11:05:32 on dist-test-slave-trms"
server_key: "40362531da3fe4603ec5074661f19c84"
server_key_iv: "6b710bc8dc4c1304c1d2603d03555150"
server_key_version: "encryptionkey@0"
I20260925 11:05:32.550981 13127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:32.566983 13127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:32.570744 13127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:32.570999 13127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:32.586612 13127 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44177
I20260925 11:05:32.586634 13193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44177 every 8 connection(s)
I20260925 11:05:32.587952 13127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:32.591646 13194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:32.596014 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 13127
I20260925 11:05:32.596240 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:32.597023 25215 external_mini_cluster.cc:1428] Setting key 6a1c0f1bf015ce4a14ef2d6c4bdbb6ae
I20260925 11:05:32.604228 13194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0: Bootstrap starting.
I20260925 11:05:32.607818 13194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:32.609112 13194 log.cc:824] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:32.611929 13194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0: No bootstrap required, opened a new log
Sep 25 11:05:32 dist-test-slave-trms krb5kdc[13114](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 11:05:32.621767 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:32.598914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57021 (local address 127.0.0.1:44177)
0925 11:05:32.599712 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:32.599728 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:32.599779 (+ 51us) server_negotiation.cc:407] Connection header received
0925 11:05:32.602506 (+ 2727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:32.602535 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:32.603062 (+ 527us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:32.603498 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:32.604732 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:32.606754 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:32.608007 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:32.609028 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:32.609229 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:32.613309 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:32.613319 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:32.613338 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:32.613388 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:32.617269 (+ 3881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:32.617992 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:32.618003 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:32.618013 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:32.618246 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:32.618697 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:32.618702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:32.618706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:32.619393 (+ 687us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:32.619732 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:32.620032 (+ 300us) server_negotiation.cc:299] Negotiation successful
0925 11:05:32.620394 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":345,"threads_started":1}
I20260925 11:05:32.622125 13194 raft_consensus.cc:348] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c728af672e0741fba2d29553ff198dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260925 11:05:32.622530 13194 raft_consensus.cc:374] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:32.622627 13194 raft_consensus.cc:729] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c728af672e0741fba2d29553ff198dd0, State: Initialized, Role: FOLLOWER
I20260925 11:05:32.623274 13194 consensus_queue.cc:260] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c728af672e0741fba2d29553ff198dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260925 11:05:32.623551 13194 raft_consensus.cc:388] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:32.623694 13194 raft_consensus.cc:482] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:32.623862 13194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:32.625980 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:32.626758 13194 raft_consensus.cc:504] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c728af672e0741fba2d29553ff198dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260925 11:05:32.627319 13194 leader_election.cc:302] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c728af672e0741fba2d29553ff198dd0; no voters:
I20260925 11:05:32.627878 13194 leader_election.cc:288] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:32.627998 13199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:32.629065 13199 raft_consensus.cc:686] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [term 1 LEADER]: Becoming Leader. State: Replica: c728af672e0741fba2d29553ff198dd0, State: Running, Role: LEADER
I20260925 11:05:32.629673 13199 consensus_queue.cc:237] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c728af672e0741fba2d29553ff198dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } }
I20260925 11:05:32.630064 13194 sys_catalog.cc:564] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:32.632236 13201 sys_catalog.cc:455] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c728af672e0741fba2d29553ff198dd0. Latest consensus state: current_term: 1 leader_uuid: "c728af672e0741fba2d29553ff198dd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c728af672e0741fba2d29553ff198dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } } }
I20260925 11:05:32.632313 13200 sys_catalog.cc:455] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c728af672e0741fba2d29553ff198dd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c728af672e0741fba2d29553ff198dd0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44177 } } }
I20260925 11:05:32.632647 13200 sys_catalog.cc:458] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:32.632647 13201 sys_catalog.cc:458] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:32.633482 13210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:32.648942 13210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:32.652030 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:32.631645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41865 (local address 127.0.0.1:44177)
0925 11:05:32.631809 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:32.631814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:32.634171 (+ 2357us) server_negotiation.cc:407] Connection header received
0925 11:05:32.634377 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:32.634383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:32.634471 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:32.634576 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:32.636638 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:32.637859 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:32.639879 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:32.640562 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:32.640666 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:32.644126 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:32.644133 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:32.644138 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:32.644178 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:32.647993 (+ 3815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:32.649040 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:32.649046 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:32.649051 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:32.649221 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:32.649848 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:32.649854 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:32.649858 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:32.650322 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:32.650607 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:32.651367 (+ 760us) server_negotiation.cc:299] Negotiation successful
0925 11:05:32.651595 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260925 11:05:32.664439 13210 catalog_manager.cc:1347] Generated new cluster ID: 43ff47ccfe5a41c685124044b0607e39
I20260925 11:05:32.664577 13210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:32.678802 13210 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:32.680227 13210 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:32.685860 13210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c728af672e0741fba2d29553ff198dd0: Generated new TSK 0
I20260925 11:05:32.687027 13210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:32.701259 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13127
I20260925 11:05:32.702490 13189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:32.793115 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 13127
W20260925 11:05:32.806711 13112 connection.cc:535] client connection to 127.0.0.1:44177 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44177): Connection reset by peer (error 104)
2026-09-25T11:05:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1867 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:32 dist-test-slave-trms krb5kdc[13226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:32 dist-test-slave-trms krb5kdc[13226](info): set up 2 sockets
Sep 25 11:05:32 dist-test-slave-trms krb5kdc[13226](info): commencing operation
krb5kdc: starting...
W20260925 11:05:33.802585 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:33 dist-test-slave-trms krb5kdc[13226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:34.037099 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:34.354616 13239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:34.354898 13239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:34.354985 13239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:34.362286 13239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:34.362402 13239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:34.362447 13239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:34.362505 13239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:34.362548 13239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:34.372599 13239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43051
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:34.373900 13239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:34.375200 13239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:34.386620 13245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:34.386727 13247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:34.386735 13244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:34.387537 13239 server_base.cc:1034] running on GCE node
I20260925 11:05:34.388299 13239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:34.389698 13239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:34.390892 13239 hybrid_clock.cc:648] HybridClock initialized: now 1790334334390845 us; error 45 us; skew 500 ppm
I20260925 11:05:34.391237 13239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:34 dist-test-slave-trms krb5kdc[13226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334334, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:34.394284 13239 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:34.395630 13239 webserver.cc:460] Webserver started at http://127.0.0.1:45163/ using document root <none> and password file <none>
I20260925 11:05:34.396270 13239 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:34.396368 13239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:34.396603 13239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:34.399551 13239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "21059fc5a4904e99a501dafdf1dcba60"
format_stamp: "Formatted at 2026-09-25 11:05:34 on dist-test-slave-trms"
server_key: "bf9ca55a16a585f679237df9490b5b37"
server_key_iv: "c19ac56d9fb9801698d8782eb7598897"
server_key_version: "encryptionkey@0"
I20260925 11:05:34.400270 13239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "21059fc5a4904e99a501dafdf1dcba60"
format_stamp: "Formatted at 2026-09-25 11:05:34 on dist-test-slave-trms"
server_key: "bf9ca55a16a585f679237df9490b5b37"
server_key_iv: "c19ac56d9fb9801698d8782eb7598897"
server_key_version: "encryptionkey@0"
I20260925 11:05:34.405351 13239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 11:05:34.408859 13254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:34.410254 13239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:05:34.410424 13239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "21059fc5a4904e99a501dafdf1dcba60"
format_stamp: "Formatted at 2026-09-25 11:05:34 on dist-test-slave-trms"
server_key: "bf9ca55a16a585f679237df9490b5b37"
server_key_iv: "c19ac56d9fb9801698d8782eb7598897"
server_key_version: "encryptionkey@0"
I20260925 11:05:34.410610 13239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:34.439005 13239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:34.444218 13239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:34.444502 13239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:34.460850 13239 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43051
I20260925 11:05:34.460819 13305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43051 every 8 connection(s)
I20260925 11:05:34.462329 13239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:34.466125 13306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:34.469226 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 13239
I20260925 11:05:34.469391 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:34.469806 25215 external_mini_cluster.cc:1428] Setting key 95b68f703c8fafdc530957d36321711d
I20260925 11:05:34.477941 13306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60: Bootstrap starting.
I20260925 11:05:34.481578 13306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:34.482797 13306 log.cc:824] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:34.483165 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:34.471638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39549 (local address 127.0.0.1:43051)
0925 11:05:34.472362 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:34.472375 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:34.472416 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:05:34.474064 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:34.474087 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:34.474565 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:34.474908 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:34.475492 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:34.476825 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:34.478641 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:34.479337 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:34.479488 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:34.480781 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:34.480789 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:34.480803 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:34.481364 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:34.481444 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:34.481469 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:34.481594 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:34.481914 (+ 320us) server_negotiation.cc:299] Negotiation successful
0925 11:05:34.482118 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":317,"threads_started":1}
I20260925 11:05:34.485915 13306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60: No bootstrap required, opened a new log
I20260925 11:05:34.487246 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:34.493976 13306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21059fc5a4904e99a501dafdf1dcba60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } }
I20260925 11:05:34.494392 13306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:34.494483 13306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21059fc5a4904e99a501dafdf1dcba60, State: Initialized, Role: FOLLOWER
I20260925 11:05:34.495108 13306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21059fc5a4904e99a501dafdf1dcba60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } }
I20260925 11:05:34.495409 13306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:34.495551 13306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:34.495739 13306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:34.497548 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:34.491357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40329 (local address 127.0.0.1:43051)
0925 11:05:34.491525 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:34.491531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:34.491894 (+ 363us) server_negotiation.cc:407] Connection header received
0925 11:05:34.492066 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:34.492070 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:34.492137 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:34.492226 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:34.492724 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:34.493484 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:34.494924 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:34.495571 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:34.495701 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:34.496785 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:34.496792 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:34.496799 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:34.496821 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:34.496866 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:34.496888 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:34.496998 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:34.497165 (+ 167us) server_negotiation.cc:299] Negotiation successful
0925 11:05:34.497222 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:05:34.498803 13306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21059fc5a4904e99a501dafdf1dcba60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } }
I20260925 11:05:34.499341 13306 leader_election.cc:302] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21059fc5a4904e99a501dafdf1dcba60; no voters:
I20260925 11:05:34.499931 13306 leader_election.cc:288] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:34.500070 13316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:34.500687 13316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [term 1 LEADER]: Becoming Leader. State: Replica: 21059fc5a4904e99a501dafdf1dcba60, State: Running, Role: LEADER
I20260925 11:05:34.501204 13316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21059fc5a4904e99a501dafdf1dcba60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } }
I20260925 11:05:34.501917 13306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:34.503895 13317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21059fc5a4904e99a501dafdf1dcba60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21059fc5a4904e99a501dafdf1dcba60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } } }
I20260925 11:05:34.504280 13317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:34.503875 13318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21059fc5a4904e99a501dafdf1dcba60. Latest consensus state: current_term: 1 leader_uuid: "21059fc5a4904e99a501dafdf1dcba60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21059fc5a4904e99a501dafdf1dcba60" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43051 } } }
I20260925 11:05:34.510694 13318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:34.511412 13324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:34.516868 13324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:34.526085 13324 catalog_manager.cc:1347] Generated new cluster ID: 29becdf4c3654cbfabaebc1d2859e142
I20260925 11:05:34.526249 13324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:34.544423 13324 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:34.545746 13324 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:34.557996 13324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21059fc5a4904e99a501dafdf1dcba60: Generated new TSK 0
I20260925 11:05:34.559120 13324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:34.568996 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13239
W20260925 11:05:34.570238 13301 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39549
W20260925 11:05:34.570967 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13239: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:34.571190 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 13239
W20260925 11:05:34.584561 13224 connection.cc:535] client connection to 127.0.0.1:43051 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43051): Connection reset by peer (error 104)
2026-09-25T11:05:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1777 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:34 dist-test-slave-trms krb5kdc[13337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:34 dist-test-slave-trms krb5kdc[13337](info): set up 2 sockets
Sep 25 11:05:34 dist-test-slave-trms krb5kdc[13337](info): commencing operation
krb5kdc: starting...
W20260925 11:05:35.586555 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.984s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:35 dist-test-slave-trms krb5kdc[13337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:35.820685 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46675
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:36.141469 13350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:36.141759 13350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:36.141892 13350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:36.149060 13350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:36.149197 13350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:36.149253 13350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:36.149345 13350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:36.149405 13350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:36.159359 13350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46675
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:36.160699 13350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:36.161999 13350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:36.173123 13355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:36.173162 13356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:36.173187 13358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:36.174088 13350 server_base.cc:1034] running on GCE node
I20260925 11:05:36.174880 13350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:36.176319 13350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:36.177528 13350 hybrid_clock.cc:648] HybridClock initialized: now 1790334336177482 us; error 46 us; skew 500 ppm
I20260925 11:05:36.177863 13350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:36 dist-test-slave-trms krb5kdc[13337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334336, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:36.180927 13350 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:36.182442 13350 webserver.cc:460] Webserver started at http://127.0.0.1:44857/ using document root <none> and password file <none>
I20260925 11:05:36.183097 13350 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:36.183211 13350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:36.183506 13350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:36.186450 13350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "7169a4420abc4c44b3166d750d3d7f8c"
format_stamp: "Formatted at 2026-09-25 11:05:36 on dist-test-slave-trms"
server_key: "9d4dc038d8e22e3c6608cea5359297e4"
server_key_iv: "f33efacf71a8ee9c367d69ac8094a068"
server_key_version: "encryptionkey@0"
I20260925 11:05:36.187171 13350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "7169a4420abc4c44b3166d750d3d7f8c"
format_stamp: "Formatted at 2026-09-25 11:05:36 on dist-test-slave-trms"
server_key: "9d4dc038d8e22e3c6608cea5359297e4"
server_key_iv: "f33efacf71a8ee9c367d69ac8094a068"
server_key_version: "encryptionkey@0"
I20260925 11:05:36.192188 13350 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:05:36.195580 13365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:36.196913 13350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:05:36.197077 13350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "7169a4420abc4c44b3166d750d3d7f8c"
format_stamp: "Formatted at 2026-09-25 11:05:36 on dist-test-slave-trms"
server_key: "9d4dc038d8e22e3c6608cea5359297e4"
server_key_iv: "f33efacf71a8ee9c367d69ac8094a068"
server_key_version: "encryptionkey@0"
I20260925 11:05:36.197253 13350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:36.202982 13350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:36.207172 13350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:36.207453 13350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:36.224321 13350 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46675
I20260925 11:05:36.224357 13416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46675 every 8 connection(s)
I20260925 11:05:36.225670 13350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:36.229511 13417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:36.233240 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 13350
I20260925 11:05:36.233407 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:36.233902 25215 external_mini_cluster.cc:1428] Setting key b767ea12f2c804164c22e48f1fb8bdce
I20260925 11:05:36.241240 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:36.235666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46189 (local address 127.0.0.1:46675)
0925 11:05:36.236417 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:36.236429 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:36.236470 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:05:36.238022 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:36.238043 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:36.238469 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:36.238797 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:36.239756 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:36.239765 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:36.239778 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:36.239792 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:36.239852 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:36.239871 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:36.240039 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:36.240141 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 11:05:36.240317 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":333,"threads_started":1}
I20260925 11:05:36.241796 13417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c: Bootstrap starting.
I20260925 11:05:36.244657 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:36.244825 13417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:36.245987 13417 log.cc:824] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:36.249046 13417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c: No bootstrap required, opened a new log
I20260925 11:05:36.251574 13420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:36.249374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50363 (local address 127.0.0.1:46675)
0925 11:05:36.249548 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:36.249554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:36.249813 (+ 259us) server_negotiation.cc:407] Connection header received
0925 11:05:36.249976 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:36.249981 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:36.250030 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:36.250112 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:36.250956 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:36.250962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:36.250970 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:36.250989 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:36.251024 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:36.251038 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:36.251133 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:36.251249 (+ 116us) server_negotiation.cc:299] Negotiation successful
0925 11:05:36.251312 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 11:05:36.256485 13417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7169a4420abc4c44b3166d750d3d7f8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } }
I20260925 11:05:36.256816 13417 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:36.256910 13417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7169a4420abc4c44b3166d750d3d7f8c, State: Initialized, Role: FOLLOWER
I20260925 11:05:36.257504 13417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7169a4420abc4c44b3166d750d3d7f8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } }
I20260925 11:05:36.257762 13417 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:36.257893 13417 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:36.258044 13417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:36.260087 13417 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7169a4420abc4c44b3166d750d3d7f8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } }
I20260925 11:05:36.260567 13417 leader_election.cc:302] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7169a4420abc4c44b3166d750d3d7f8c; no voters:
I20260925 11:05:36.261082 13417 leader_election.cc:288] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:36.261207 13427 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:36.261785 13427 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [term 1 LEADER]: Becoming Leader. State: Replica: 7169a4420abc4c44b3166d750d3d7f8c, State: Running, Role: LEADER
I20260925 11:05:36.262326 13427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7169a4420abc4c44b3166d750d3d7f8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } }
I20260925 11:05:36.263023 13417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:36.264781 13429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7169a4420abc4c44b3166d750d3d7f8c. Latest consensus state: current_term: 1 leader_uuid: "7169a4420abc4c44b3166d750d3d7f8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7169a4420abc4c44b3166d750d3d7f8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } } }
I20260925 11:05:36.265101 13429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:36.265530 13428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7169a4420abc4c44b3166d750d3d7f8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7169a4420abc4c44b3166d750d3d7f8c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46675 } } }
I20260925 11:05:36.265836 13428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:36.266027 13433 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:36.272652 13433 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:36.283509 13433 catalog_manager.cc:1347] Generated new cluster ID: 35701fc266f440fc8b74ee7af11b29ef
I20260925 11:05:36.283694 13433 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:36.296892 13433 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:36.297290 13433 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:36.306630 13433 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7169a4420abc4c44b3166d750d3d7f8c: Generated new TSK 0
I20260925 11:05:36.307420 13433 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:36.317401 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13350
W20260925 11:05:36.318527 13412 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46189
W20260925 11:05:36.319154 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13350: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:36.319280 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 13350
2026-09-25T11:05:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1747 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:36 dist-test-slave-trms krb5kdc[13449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:36 dist-test-slave-trms krb5kdc[13449](info): set up 2 sockets
Sep 25 11:05:36 dist-test-slave-trms krb5kdc[13449](info): commencing operation
krb5kdc: starting...
W20260925 11:05:37.338555 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:37 dist-test-slave-trms krb5kdc[13449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:37.575660 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:37.893417 13462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:37.893711 13462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:37.893853 13462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:37.900981 13462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:37.901099 13462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:37.901180 13462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:37.901271 13462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:37.901330 13462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:37.910955 13462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:37.912178 13462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:37.913444 13462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:37.924495 13467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:37.924420 13468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:37.924451 13470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:37.925429 13462 server_base.cc:1034] running on GCE node
I20260925 11:05:37.926167 13462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:37.927546 13462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:37.928735 13462 hybrid_clock.cc:648] HybridClock initialized: now 1790334337928681 us; error 42 us; skew 500 ppm
I20260925 11:05:37.929039 13462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:37 dist-test-slave-trms krb5kdc[13449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334337, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:37.935257 13462 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:37.936667 13462 webserver.cc:460] Webserver started at http://127.0.0.1:40395/ using document root <none> and password file <none>
I20260925 11:05:37.937317 13462 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:37.937425 13462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:37.937711 13462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:37.940698 13462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "5cabc7e60867456ab77fe11cab1e8289"
format_stamp: "Formatted at 2026-09-25 11:05:37 on dist-test-slave-trms"
server_key: "9e8f6cde4c376749925203f878ca5596"
server_key_iv: "c175342e636f7f479de9b3f71fdc1809"
server_key_version: "encryptionkey@0"
I20260925 11:05:37.941442 13462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "5cabc7e60867456ab77fe11cab1e8289"
format_stamp: "Formatted at 2026-09-25 11:05:37 on dist-test-slave-trms"
server_key: "9e8f6cde4c376749925203f878ca5596"
server_key_iv: "c175342e636f7f479de9b3f71fdc1809"
server_key_version: "encryptionkey@0"
I20260925 11:05:37.946722 13462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260925 11:05:37.950307 13477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:37.951682 13462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:05:37.951846 13462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "5cabc7e60867456ab77fe11cab1e8289"
format_stamp: "Formatted at 2026-09-25 11:05:37 on dist-test-slave-trms"
server_key: "9e8f6cde4c376749925203f878ca5596"
server_key_iv: "c175342e636f7f479de9b3f71fdc1809"
server_key_version: "encryptionkey@0"
I20260925 11:05:37.952034 13462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:37.968408 13462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:37.972404 13462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:37.972651 13462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:37.989530 13462 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39931
I20260925 11:05:37.989549 13528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39931 every 8 connection(s)
I20260925 11:05:37.990875 13462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:37.994541 13529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:37.998365 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 13462
I20260925 11:05:37.998549 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:37.999080 25215 external_mini_cluster.cc:1428] Setting key b4a546f4661d4d63b87829d252e07fbc
I20260925 11:05:38.007208 13529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289: Bootstrap starting.
I20260925 11:05:38.010948 13529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:38.012261 13529 log.cc:824] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289: Log is configured to *not* fsync() on all Append() calls
Sep 25 11:05:38 dist-test-slave-trms krb5kdc[13449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 11:05:38.015247 13529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289: No bootstrap required, opened a new log
I20260925 11:05:38.021343 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:38.000796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50545 (local address 127.0.0.1:39931)
0925 11:05:38.001530 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:38.001542 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:38.001582 (+ 40us) server_negotiation.cc:407] Connection header received
0925 11:05:38.003206 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:38.003229 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:38.003660 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:38.003993 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:38.005719 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:38.007060 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:38.008773 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:38.009486 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:38.009610 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:38.014362 (+ 4752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:38.014370 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:38.014390 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:38.014425 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:38.017359 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:38.018045 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:38.018051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:38.018058 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:38.018265 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:38.018616 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:38.018621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:38.018624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:38.019166 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:38.019480 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:38.019902 (+ 422us) server_negotiation.cc:299] Negotiation successful
0925 11:05:38.020330 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":344,"threads_started":1}
I20260925 11:05:38.022845 13529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cabc7e60867456ab77fe11cab1e8289" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } }
I20260925 11:05:38.023185 13529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:38.023276 13529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cabc7e60867456ab77fe11cab1e8289, State: Initialized, Role: FOLLOWER
I20260925 11:05:38.023851 13529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cabc7e60867456ab77fe11cab1e8289" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } }
I20260925 11:05:38.024127 13529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:38.024272 13529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:38.024467 13529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:38.024972 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:38.027549 13529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cabc7e60867456ab77fe11cab1e8289" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } }
I20260925 11:05:38.028111 13529 leader_election.cc:302] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cabc7e60867456ab77fe11cab1e8289; no voters:
I20260925 11:05:38.028852 13529 leader_election.cc:288] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:38.029174 13535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:38.032676 13535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [term 1 LEADER]: Becoming Leader. State: Replica: 5cabc7e60867456ab77fe11cab1e8289, State: Running, Role: LEADER
I20260925 11:05:38.033459 13535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cabc7e60867456ab77fe11cab1e8289" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } }
I20260925 11:05:38.034418 13529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:38.038558 13537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cabc7e60867456ab77fe11cab1e8289. Latest consensus state: current_term: 1 leader_uuid: "5cabc7e60867456ab77fe11cab1e8289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cabc7e60867456ab77fe11cab1e8289" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } } }
I20260925 11:05:38.038940 13537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:38.039428 13536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cabc7e60867456ab77fe11cab1e8289" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cabc7e60867456ab77fe11cab1e8289" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39931 } } }
I20260925 11:05:38.039726 13536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:38.039889 13545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:38.050092 13545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:38.056308 13532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:38.034048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38819 (local address 127.0.0.1:39931)
0925 11:05:38.034262 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:38.034268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:38.038997 (+ 4729us) server_negotiation.cc:407] Connection header received
0925 11:05:38.039177 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:38.039183 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:38.039264 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:38.039363 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:38.041366 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:38.042556 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:38.044507 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:38.045249 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:38.045351 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:38.048512 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:38.048519 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:38.048524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:38.048569 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:38.052100 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:38.052922 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:38.052927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:38.052930 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:38.053074 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:38.053569 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:38.053574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:38.053577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:38.054085 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:38.054368 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:38.055657 (+ 1289us) server_negotiation.cc:299] Negotiation successful
0925 11:05:38.055902 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:05:38.065266 13545 catalog_manager.cc:1347] Generated new cluster ID: 57837e1c794048508d946f627918eabb
I20260925 11:05:38.065382 13545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:38.098068 13545 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:38.099433 13545 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:38.106698 13545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cabc7e60867456ab77fe11cab1e8289: Generated new TSK 0
I20260925 11:05:38.107784 13545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:38.123194 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13462
I20260925 11:05:38.124231 13524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:38.215564 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 13462
2026-09-25T11:05:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1896 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:38 dist-test-slave-trms krb5kdc[13561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:38 dist-test-slave-trms krb5kdc[13561](info): set up 2 sockets
Sep 25 11:05:38 dist-test-slave-trms krb5kdc[13561](info): commencing operation
krb5kdc: starting...
W20260925 11:05:39.218493 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:39 dist-test-slave-trms krb5kdc[13561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:39.455266 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:39.772485 13574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:39.772763 13574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:39.772874 13574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:39.780026 13574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:39.780138 13574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:39.780217 13574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:39.780298 13574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:39.780359 13574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:39.790060 13574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:39.791345 13574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:39.792650 13574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:39.803850 13579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:39.803910 13582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:39.803882 13580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:39.804626 13574 server_base.cc:1034] running on GCE node
I20260925 11:05:39.805398 13574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:39.806838 13574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:39.808030 13574 hybrid_clock.cc:648] HybridClock initialized: now 1790334339807987 us; error 36 us; skew 500 ppm
I20260925 11:05:39.808362 13574 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:39 dist-test-slave-trms krb5kdc[13561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334339, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:39.811352 13574 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:39.812697 13574 webserver.cc:460] Webserver started at http://127.0.0.1:43369/ using document root <none> and password file <none>
I20260925 11:05:39.813342 13574 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:39.813457 13574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:39.813720 13574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:39.816587 13574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "b5fc9547c70d43399a2c4e511a4dc268"
format_stamp: "Formatted at 2026-09-25 11:05:39 on dist-test-slave-trms"
server_key: "bbb164a6ec0f84425f6bc9b96ceb823b"
server_key_iv: "6a0190d2686a62bd5028d6d5f2f89cd7"
server_key_version: "encryptionkey@0"
I20260925 11:05:39.817301 13574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "b5fc9547c70d43399a2c4e511a4dc268"
format_stamp: "Formatted at 2026-09-25 11:05:39 on dist-test-slave-trms"
server_key: "bbb164a6ec0f84425f6bc9b96ceb823b"
server_key_iv: "6a0190d2686a62bd5028d6d5f2f89cd7"
server_key_version: "encryptionkey@0"
I20260925 11:05:39.822170 13574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260925 11:05:39.825495 13589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:39.826776 13574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260925 11:05:39.826934 13574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "b5fc9547c70d43399a2c4e511a4dc268"
format_stamp: "Formatted at 2026-09-25 11:05:39 on dist-test-slave-trms"
server_key: "bbb164a6ec0f84425f6bc9b96ceb823b"
server_key_iv: "6a0190d2686a62bd5028d6d5f2f89cd7"
server_key_version: "encryptionkey@0"
I20260925 11:05:39.827126 13574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:39.844826 13574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:39.848727 13574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:39.848974 13574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:39.865566 13574 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35717
I20260925 11:05:39.865584 13640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35717 every 8 connection(s)
I20260925 11:05:39.866937 13574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:39.867898 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 13574
I20260925 11:05:39.868090 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:39.868621 25215 external_mini_cluster.cc:1428] Setting key 919b4e8cc625ae687541e39346c1a811
I20260925 11:05:39.871330 13641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:39.881266 13644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:39.870401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33481 (local address 127.0.0.1:35717)
0925 11:05:39.871107 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:39.871119 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:39.871160 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:05:39.872733 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:39.872757 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:39.873130 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:39.873452 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:39.874000 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:39.875353 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:39.877090 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:39.877726 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:39.877845 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:39.879005 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:39.879013 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:39.879027 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:39.879568 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:39.879645 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:39.879673 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:39.879764 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:39.879988 (+ 224us) server_negotiation.cc:299] Negotiation successful
0925 11:05:39.880262 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":318,"threads_started":1}
I20260925 11:05:39.883008 13641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268: Bootstrap starting.
I20260925 11:05:39.884635 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:39.886435 13641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:39.887705 13641 log.cc:824] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:39.890884 13641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268: No bootstrap required, opened a new log
I20260925 11:05:39.895926 13644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:39.889768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51251 (local address 127.0.0.1:35717)
0925 11:05:39.889945 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:39.889951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:39.890313 (+ 362us) server_negotiation.cc:407] Connection header received
0925 11:05:39.890486 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:39.890492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:39.890567 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:39.890675 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:39.891150 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:39.892203 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:39.893410 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:39.894116 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:39.894277 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:39.895216 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:39.895222 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:39.895226 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:39.895247 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:39.895294 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:39.895313 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:39.895406 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:39.895479 (+ 73us) server_negotiation.cc:299] Negotiation successful
0925 11:05:39.895576 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260925 11:05:39.899395 13641 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fc9547c70d43399a2c4e511a4dc268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35717 } }
I20260925 11:05:39.899766 13641 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:39.899864 13641 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5fc9547c70d43399a2c4e511a4dc268, State: Initialized, Role: FOLLOWER
I20260925 11:05:39.900498 13641 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fc9547c70d43399a2c4e511a4dc268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35717 } }
I20260925 11:05:39.900703 13641 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:39.900804 13641 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:39.900923 13641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:39.903295 13641 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fc9547c70d43399a2c4e511a4dc268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35717 } }
I20260925 11:05:39.903817 13641 leader_election.cc:302] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5fc9547c70d43399a2c4e511a4dc268; no voters:
I20260925 11:05:39.904300 13641 leader_election.cc:288] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:39.904412 13651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:39.905040 13651 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [term 1 LEADER]: Becoming Leader. State: Replica: b5fc9547c70d43399a2c4e511a4dc268, State: Running, Role: LEADER
I20260925 11:05:39.905542 13651 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fc9547c70d43399a2c4e511a4dc268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35717 } }
I20260925 11:05:39.906298 13641 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:39.907984 13652 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5fc9547c70d43399a2c4e511a4dc268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fc9547c70d43399a2c4e511a4dc268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35717 } } }
I20260925 11:05:39.908402 13652 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:39.909823 13653 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5fc9547c70d43399a2c4e511a4dc268. Latest consensus state: current_term: 1 leader_uuid: "b5fc9547c70d43399a2c4e511a4dc268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5fc9547c70d43399a2c4e511a4dc268" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35717 } } }
I20260925 11:05:39.910085 13653 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:39.910552 13659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:39.915807 13659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:39.924587 13659 catalog_manager.cc:1347] Generated new cluster ID: 060d94475a364177a67450cbb60c0d27
I20260925 11:05:39.924702 13659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:39.939291 13659 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:39.940560 13659 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:39.946830 13659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5fc9547c70d43399a2c4e511a4dc268: Generated new TSK 0
I20260925 11:05:39.947767 13659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:39.960884 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13574
W20260925 11:05:39.961961 13636 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:33481
W20260925 11:05:39.962682 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13574: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:39.962816 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 13574
2026-09-25T11:05:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1746 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:40 dist-test-slave-trms krb5kdc[13672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:40 dist-test-slave-trms krb5kdc[13672](info): set up 2 sockets
Sep 25 11:05:40 dist-test-slave-trms krb5kdc[13672](info): commencing operation
krb5kdc: starting...
W20260925 11:05:40.970712 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.005s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:41 dist-test-slave-trms krb5kdc[13672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:41.207710 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43371
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:41.526165 13685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:41.526479 13685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:41.526573 13685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:41.533633 13685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:41.533746 13685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:41.533800 13685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:41.533882 13685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:41.533936 13685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:41.543571 13685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:41.544832 13685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:41.546125 13685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:41.556885 13691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:41.557008 13690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:41.557274 13693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:41.557973 13685 server_base.cc:1034] running on GCE node
I20260925 11:05:41.558797 13685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:41.560155 13685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:41.561367 13685 hybrid_clock.cc:648] HybridClock initialized: now 1790334341561315 us; error 39 us; skew 500 ppm
I20260925 11:05:41.561713 13685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:41 dist-test-slave-trms krb5kdc[13672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334341, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:41.564682 13685 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:41.566136 13685 webserver.cc:460] Webserver started at http://127.0.0.1:43701/ using document root <none> and password file <none>
I20260925 11:05:41.566807 13685 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:41.566919 13685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:41.567219 13685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:41.570093 13685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "57f416b060cc455290589d7d4e342c55"
format_stamp: "Formatted at 2026-09-25 11:05:41 on dist-test-slave-trms"
server_key: "d16812f131bd783c5702dec737f082c3"
server_key_iv: "7ae32be5121d512b428373f422f749da"
server_key_version: "encryptionkey@0"
I20260925 11:05:41.570889 13685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "57f416b060cc455290589d7d4e342c55"
format_stamp: "Formatted at 2026-09-25 11:05:41 on dist-test-slave-trms"
server_key: "d16812f131bd783c5702dec737f082c3"
server_key_iv: "7ae32be5121d512b428373f422f749da"
server_key_version: "encryptionkey@0"
I20260925 11:05:41.575894 13685 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:05:41.579257 13700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:41.580487 13685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:05:41.580636 13685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "57f416b060cc455290589d7d4e342c55"
format_stamp: "Formatted at 2026-09-25 11:05:41 on dist-test-slave-trms"
server_key: "d16812f131bd783c5702dec737f082c3"
server_key_iv: "7ae32be5121d512b428373f422f749da"
server_key_version: "encryptionkey@0"
I20260925 11:05:41.580814 13685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:41.586499 13685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:41.590636 13685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:41.590909 13685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:41.607174 13685 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43371
I20260925 11:05:41.607239 13751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43371 every 8 connection(s)
I20260925 11:05:41.608492 13685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:41.609699 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 13685
I20260925 11:05:41.609885 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:41.610456 25215 external_mini_cluster.cc:1428] Setting key fb4238db1b9752167d28f4ed1ddaa8e9
I20260925 11:05:41.612802 13752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:41.617691 13755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:41.612138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48469 (local address 127.0.0.1:43371)
0925 11:05:41.612891 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:41.612907 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:41.612955 (+ 48us) server_negotiation.cc:407] Connection header received
0925 11:05:41.614563 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:41.614584 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:41.614917 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:41.615248 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:41.616221 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:41.616227 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:41.616239 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:41.616256 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:41.616335 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:41.616359 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:41.616524 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:41.616617 (+ 93us) server_negotiation.cc:299] Negotiation successful
0925 11:05:41.616785 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":314,"threads_started":1}
I20260925 11:05:41.621224 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:41.625053 13752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55: Bootstrap starting.
I20260925 11:05:41.628301 13755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:41.626148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59703 (local address 127.0.0.1:43371)
0925 11:05:41.626345 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:41.626350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:41.626606 (+ 256us) server_negotiation.cc:407] Connection header received
0925 11:05:41.626762 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:41.626765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:41.626812 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:41.626901 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:41.627733 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:41.627737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:41.627741 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:41.627754 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:41.627790 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:41.627806 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:41.627896 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:41.628010 (+ 114us) server_negotiation.cc:299] Negotiation successful
0925 11:05:41.628051 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260925 11:05:41.628674 13752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:41.629824 13752 log.cc:824] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:41.632350 13752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55: No bootstrap required, opened a new log
I20260925 11:05:41.639216 13752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f416b060cc455290589d7d4e342c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } }
I20260925 11:05:41.639544 13752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:41.639612 13752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f416b060cc455290589d7d4e342c55, State: Initialized, Role: FOLLOWER
I20260925 11:05:41.640162 13752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f416b060cc455290589d7d4e342c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } }
I20260925 11:05:41.640400 13752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:41.640554 13752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:41.640748 13752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:41.642843 13752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f416b060cc455290589d7d4e342c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } }
I20260925 11:05:41.643288 13752 leader_election.cc:302] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57f416b060cc455290589d7d4e342c55; no voters:
I20260925 11:05:41.643803 13752 leader_election.cc:288] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:41.643929 13762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:41.644505 13762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [term 1 LEADER]: Becoming Leader. State: Replica: 57f416b060cc455290589d7d4e342c55, State: Running, Role: LEADER
I20260925 11:05:41.645027 13762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f416b060cc455290589d7d4e342c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } }
I20260925 11:05:41.645684 13752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:41.647852 13764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57f416b060cc455290589d7d4e342c55. Latest consensus state: current_term: 1 leader_uuid: "57f416b060cc455290589d7d4e342c55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f416b060cc455290589d7d4e342c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } } }
I20260925 11:05:41.647861 13763 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57f416b060cc455290589d7d4e342c55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f416b060cc455290589d7d4e342c55" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43371 } } }
I20260925 11:05:41.648322 13763 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:41.649677 13767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:41.649688 13764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:41.654942 13767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:41.664319 13767 catalog_manager.cc:1347] Generated new cluster ID: dae2a757f51c4d67ac5771411ee03ad4
I20260925 11:05:41.664438 13767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:41.682349 13767 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:41.682756 13767 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:41.694597 13767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57f416b060cc455290589d7d4e342c55: Generated new TSK 0
I20260925 11:05:41.695405 13767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:41.703790 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13685
W20260925 11:05:41.704877 13747 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48469
W20260925 11:05:41.705516 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13685: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:41.705693 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 13685
2026-09-25T11:05:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1743 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:41 dist-test-slave-trms krb5kdc[13783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:41 dist-test-slave-trms krb5kdc[13783](info): set up 2 sockets
Sep 25 11:05:41 dist-test-slave-trms krb5kdc[13783](info): commencing operation
krb5kdc: starting...
W20260925 11:05:42.718575 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:42 dist-test-slave-trms krb5kdc[13783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:42.963491 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40663
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:43.279469 13796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:43.279767 13796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:43.279894 13796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:43.287616 13796 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:43.287735 13796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:43.287822 13796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:43.287905 13796 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:43.287963 13796 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:43.297783 13796 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:43.299096 13796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:43.300402 13796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:43.311604 13801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:43.311621 13802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:43.311560 13804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:43.312330 13796 server_base.cc:1034] running on GCE node
I20260925 11:05:43.313095 13796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:43.314586 13796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:43.315786 13796 hybrid_clock.cc:648] HybridClock initialized: now 1790334343315749 us; error 31 us; skew 500 ppm
I20260925 11:05:43.316143 13796 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:43 dist-test-slave-trms krb5kdc[13783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334343, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:43.326077 13796 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:43.327610 13796 webserver.cc:460] Webserver started at http://127.0.0.1:42709/ using document root <none> and password file <none>
I20260925 11:05:43.328298 13796 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:43.328418 13796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:43.328697 13796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:43.331677 13796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "321db8cd98be4c849d64306915682123"
format_stamp: "Formatted at 2026-09-25 11:05:43 on dist-test-slave-trms"
server_key: "70e6127977a3bfb1ae55b18cac56c730"
server_key_iv: "b492944d985512aa66639f8b61cd6f5e"
server_key_version: "encryptionkey@0"
I20260925 11:05:43.332428 13796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "321db8cd98be4c849d64306915682123"
format_stamp: "Formatted at 2026-09-25 11:05:43 on dist-test-slave-trms"
server_key: "70e6127977a3bfb1ae55b18cac56c730"
server_key_iv: "b492944d985512aa66639f8b61cd6f5e"
server_key_version: "encryptionkey@0"
I20260925 11:05:43.337381 13796 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 11:05:43.340739 13811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:43.342135 13796 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:05:43.342334 13796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "321db8cd98be4c849d64306915682123"
format_stamp: "Formatted at 2026-09-25 11:05:43 on dist-test-slave-trms"
server_key: "70e6127977a3bfb1ae55b18cac56c730"
server_key_iv: "b492944d985512aa66639f8b61cd6f5e"
server_key_version: "encryptionkey@0"
I20260925 11:05:43.342509 13796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:43.356165 13796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:43.360532 13796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:43.360813 13796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:43.378420 13796 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40663
I20260925 11:05:43.378440 13862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40663 every 8 connection(s)
I20260925 11:05:43.379824 13796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:43.383592 13863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:43.385809 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 13796
I20260925 11:05:43.385994 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:43.386472 25215 external_mini_cluster.cc:1428] Setting key 5acc38535d89959b847f9ba6867ced1a
I20260925 11:05:43.394680 13863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123: Bootstrap starting.
I20260925 11:05:43.397800 13863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:43.398802 13863 log.cc:824] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123: Log is configured to *not* fsync() on all Append() calls
Sep 25 11:05:43 dist-test-slave-trms krb5kdc[13783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 11:05:43.401540 13863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123: No bootstrap required, opened a new log
I20260925 11:05:43.407744 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:43.388267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:58351 (local address 127.0.0.1:40663)
0925 11:05:43.389069 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:43.389082 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:43.389143 (+ 61us) server_negotiation.cc:407] Connection header received
0925 11:05:43.390827 (+ 1684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:43.390851 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:43.391251 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:43.391593 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:43.393043 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:43.394381 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:43.395671 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:43.396357 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:43.396550 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:43.400648 (+ 4098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:43.400656 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:43.400671 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:43.400705 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:43.403679 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:43.404427 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:43.404437 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:43.404443 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:43.404609 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:43.405063 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:43.405067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:43.405069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:43.405596 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:43.405911 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:43.406458 (+ 547us) server_negotiation.cc:299] Negotiation successful
0925 11:05:43.406746 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":328,"threads_started":1}
I20260925 11:05:43.409325 13863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321db8cd98be4c849d64306915682123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } }
I20260925 11:05:43.409746 13863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:43.409855 13863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 321db8cd98be4c849d64306915682123, State: Initialized, Role: FOLLOWER
I20260925 11:05:43.410476 13863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321db8cd98be4c849d64306915682123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } }
I20260925 11:05:43.410800 13863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:43.410982 13863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:43.411173 13863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:43.412401 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:43.414400 13863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321db8cd98be4c849d64306915682123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } }
I20260925 11:05:43.415053 13863 leader_election.cc:302] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 321db8cd98be4c849d64306915682123; no voters:
I20260925 11:05:43.415753 13863 leader_election.cc:288] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:43.416169 13868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:43.419149 13863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:43.425686 13868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [term 1 LEADER]: Becoming Leader. State: Replica: 321db8cd98be4c849d64306915682123, State: Running, Role: LEADER
I20260925 11:05:43.426409 13868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321db8cd98be4c849d64306915682123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } }
I20260925 11:05:43.429663 13870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "321db8cd98be4c849d64306915682123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321db8cd98be4c849d64306915682123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } } }
I20260925 11:05:43.430104 13870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:43.431502 13872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 321db8cd98be4c849d64306915682123. Latest consensus state: current_term: 1 leader_uuid: "321db8cd98be4c849d64306915682123" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "321db8cd98be4c849d64306915682123" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40663 } } }
I20260925 11:05:43.431821 13872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:43.436213 13883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:43.442610 13883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:43.449429 13866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:43.429318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:57219 (local address 127.0.0.1:40663)
0925 11:05:43.429498 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:43.429506 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:43.431453 (+ 1947us) server_negotiation.cc:407] Connection header received
0925 11:05:43.431859 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:43.431864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:43.431950 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:43.432035 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:43.435516 (+ 3481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:43.436479 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:43.437761 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:43.438515 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:43.438676 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:43.441960 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:43.441967 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:43.441971 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:43.442011 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:43.445721 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:43.446775 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:43.446780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:43.446783 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:43.446931 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:43.447420 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:43.447426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:43.447429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:43.447908 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:43.448178 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:43.448630 (+ 452us) server_negotiation.cc:299] Negotiation successful
0925 11:05:43.449036 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 11:05:43.456202 13883 catalog_manager.cc:1347] Generated new cluster ID: 8d8c787b258c4e398eefac39abb304c8
I20260925 11:05:43.456314 13883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:43.486213 13883 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:43.488081 13883 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:43.500751 13883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 321db8cd98be4c849d64306915682123: Generated new TSK 0
I20260925 11:05:43.501789 13883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:43.514357 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13796
I20260925 11:05:43.515645 13858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:43.607635 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 13796
W20260925 11:05:43.622819 13781 connection.cc:535] client connection to 127.0.0.1:40663 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40663): Connection reset by peer (error 104)
2026-09-25T11:05:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1903 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:43 dist-test-slave-trms krb5kdc[13896](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:43 dist-test-slave-trms krb5kdc[13896](info): set up 2 sockets
Sep 25 11:05:43 dist-test-slave-trms krb5kdc[13896](info): commencing operation
krb5kdc: starting...
W20260925 11:05:44.626479 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:44 dist-test-slave-trms krb5kdc[13896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:44.865396 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38103
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38103
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:45.182844 13909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:45.183126 13909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:45.183218 13909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:45.190423 13909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:45.190537 13909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:45.190582 13909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:45.190641 13909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:45.190677 13909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:45.200629 13909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38103
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:45.201933 13909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:45.203336 13909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:45.214664 13915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:45.214828 13917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:45.215107 13914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:45.215453 13909 server_base.cc:1034] running on GCE node
I20260925 11:05:45.216305 13909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:45.217746 13909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:45.218976 13909 hybrid_clock.cc:648] HybridClock initialized: now 1790334345218923 us; error 41 us; skew 500 ppm
I20260925 11:05:45.219322 13909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:45 dist-test-slave-trms krb5kdc[13896](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334345, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:45.222323 13909 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:45.223779 13909 webserver.cc:460] Webserver started at http://127.0.0.1:37829/ using document root <none> and password file <none>
I20260925 11:05:45.224440 13909 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:45.224555 13909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:45.224834 13909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:45.227806 13909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "45c18cba0fa6427b8592e8dfe11fb6ec"
format_stamp: "Formatted at 2026-09-25 11:05:45 on dist-test-slave-trms"
server_key: "ca4f9f1e348ca9a66d42fe0fdebb83f6"
server_key_iv: "008491b651a574114a848a74a3b9b08c"
server_key_version: "encryptionkey@0"
I20260925 11:05:45.228538 13909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "45c18cba0fa6427b8592e8dfe11fb6ec"
format_stamp: "Formatted at 2026-09-25 11:05:45 on dist-test-slave-trms"
server_key: "ca4f9f1e348ca9a66d42fe0fdebb83f6"
server_key_iv: "008491b651a574114a848a74a3b9b08c"
server_key_version: "encryptionkey@0"
I20260925 11:05:45.233552 13909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260925 11:05:45.236987 13924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:45.238384 13909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:05:45.238540 13909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "45c18cba0fa6427b8592e8dfe11fb6ec"
format_stamp: "Formatted at 2026-09-25 11:05:45 on dist-test-slave-trms"
server_key: "ca4f9f1e348ca9a66d42fe0fdebb83f6"
server_key_iv: "008491b651a574114a848a74a3b9b08c"
server_key_version: "encryptionkey@0"
I20260925 11:05:45.238713 13909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:45.258080 13909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:45.261967 13909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:45.262259 13909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:45.279541 13909 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38103
I20260925 11:05:45.279573 13975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38103 every 8 connection(s)
I20260925 11:05:45.280887 13909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:45.284628 13976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:45.287487 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 13909
I20260925 11:05:45.287653 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:45.288069 25215 external_mini_cluster.cc:1428] Setting key e065b5341ea6838c4768d425f491a9dc
I20260925 11:05:45.295876 13976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec: Bootstrap starting.
I20260925 11:05:45.299021 13976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:45.300153 13976 log.cc:824] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:45.300923 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:45.290032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:47991 (local address 127.0.0.1:38103)
0925 11:05:45.290798 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:45.290811 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:45.290853 (+ 42us) server_negotiation.cc:407] Connection header received
0925 11:05:45.292493 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:45.292515 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:45.292953 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:45.293327 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:45.293883 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:45.295236 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:45.296574 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:45.297258 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:45.297423 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:45.298560 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:45.298568 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:45.298582 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:45.299107 (+ 525us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:45.299197 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:45.299234 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:45.299385 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:45.299633 (+ 248us) server_negotiation.cc:299] Negotiation successful
0925 11:05:45.299794 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":299,"threads_started":1}
I20260925 11:05:45.303179 13976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec: No bootstrap required, opened a new log
I20260925 11:05:45.304654 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:45.311152 13976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c18cba0fa6427b8592e8dfe11fb6ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38103 } }
I20260925 11:05:45.311573 13976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:45.311668 13976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45c18cba0fa6427b8592e8dfe11fb6ec, State: Initialized, Role: FOLLOWER
I20260925 11:05:45.312235 13976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c18cba0fa6427b8592e8dfe11fb6ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38103 } }
I20260925 11:05:45.312457 13976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:45.312599 13976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:45.312774 13976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:45.314993 13979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:45.308746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:49083 (local address 127.0.0.1:38103)
0925 11:05:45.308937 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:45.308943 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:45.309409 (+ 466us) server_negotiation.cc:407] Connection header received
0925 11:05:45.309555 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:45.309561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:45.309636 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:45.309743 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:45.310174 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:45.311238 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:45.312552 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:45.313175 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:45.313302 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:45.314287 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:45.314294 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:45.314299 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:45.314320 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:45.314395 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:45.314415 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:45.314532 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:45.314636 (+ 104us) server_negotiation.cc:299] Negotiation successful
0925 11:05:45.314692 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260925 11:05:45.315706 13976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c18cba0fa6427b8592e8dfe11fb6ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38103 } }
I20260925 11:05:45.316205 13976 leader_election.cc:302] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45c18cba0fa6427b8592e8dfe11fb6ec; no voters:
I20260925 11:05:45.316707 13976 leader_election.cc:288] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:45.316819 13986 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:45.317463 13986 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [term 1 LEADER]: Becoming Leader. State: Replica: 45c18cba0fa6427b8592e8dfe11fb6ec, State: Running, Role: LEADER
I20260925 11:05:45.317965 13986 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c18cba0fa6427b8592e8dfe11fb6ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38103 } }
I20260925 11:05:45.318594 13976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:45.320633 13987 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45c18cba0fa6427b8592e8dfe11fb6ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c18cba0fa6427b8592e8dfe11fb6ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38103 } } }
I20260925 11:05:45.321024 13987 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:45.325801 13988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45c18cba0fa6427b8592e8dfe11fb6ec. Latest consensus state: current_term: 1 leader_uuid: "45c18cba0fa6427b8592e8dfe11fb6ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45c18cba0fa6427b8592e8dfe11fb6ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38103 } } }
I20260925 11:05:45.326639 13988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:45.327067 13994 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:45.332011 13994 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:45.341584 13994 catalog_manager.cc:1347] Generated new cluster ID: ed171711e6904d3e804285af543525e8
I20260925 11:05:45.341713 13994 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:45.363457 13994 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:45.364766 13994 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:45.372285 13994 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45c18cba0fa6427b8592e8dfe11fb6ec: Generated new TSK 0
I20260925 11:05:45.373114 13994 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:45.378883 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13909
W20260925 11:05:45.380059 13971 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.198:47991
W20260925 11:05:45.380755 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 13909: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:45.380893 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 13909
W20260925 11:05:45.394434 13894 connection.cc:535] client connection to 127.0.0.1:38103 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38103): Connection reset by peer (error 104)
2026-09-25T11:05:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1771 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:45 dist-test-slave-trms krb5kdc[14007](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:45 dist-test-slave-trms krb5kdc[14007](info): set up 2 sockets
Sep 25 11:05:45 dist-test-slave-trms krb5kdc[14007](info): commencing operation
krb5kdc: starting...
W20260925 11:05:46.390516 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.977s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:46 dist-test-slave-trms krb5kdc[14007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:46.643236 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40677
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:46.963706 14020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:46.963999 14020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:46.964128 14020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:46.971344 14020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:46.971472 14020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:46.971526 14020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:46.971585 14020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:46.971627 14020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:46.981518 14020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:46.982852 14020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:46.984094 14020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:46.995101 14025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:46.995064 14028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:46.995134 14026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:46.996088 14020 server_base.cc:1034] running on GCE node
I20260925 11:05:46.996850 14020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:46.998250 14020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:46.999462 14020 hybrid_clock.cc:648] HybridClock initialized: now 1790334346999415 us; error 39 us; skew 500 ppm
I20260925 11:05:46.999815 14020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:46 dist-test-slave-trms krb5kdc[14007](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334346, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:47.002782 14020 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:47.004266 14020 webserver.cc:460] Webserver started at http://127.0.0.1:39519/ using document root <none> and password file <none>
I20260925 11:05:47.004946 14020 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:47.005065 14020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:47.005348 14020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:47.008344 14020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "5edee7d9dfad4941855191d16e51b3a1"
format_stamp: "Formatted at 2026-09-25 11:05:46 on dist-test-slave-trms"
server_key: "f1babadd17f27b09ebf244cf6b79e15f"
server_key_iv: "9c1235b5b3b7ecebe9a396153440e87b"
server_key_version: "encryptionkey@0"
I20260925 11:05:47.009087 14020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "5edee7d9dfad4941855191d16e51b3a1"
format_stamp: "Formatted at 2026-09-25 11:05:46 on dist-test-slave-trms"
server_key: "f1babadd17f27b09ebf244cf6b79e15f"
server_key_iv: "9c1235b5b3b7ecebe9a396153440e87b"
server_key_version: "encryptionkey@0"
I20260925 11:05:47.014369 14020 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260925 11:05:47.017827 14035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:47.019248 14020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:05:47.019418 14020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "5edee7d9dfad4941855191d16e51b3a1"
format_stamp: "Formatted at 2026-09-25 11:05:46 on dist-test-slave-trms"
server_key: "f1babadd17f27b09ebf244cf6b79e15f"
server_key_iv: "9c1235b5b3b7ecebe9a396153440e87b"
server_key_version: "encryptionkey@0"
I20260925 11:05:47.019610 14020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:47.025110 14020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:47.028961 14020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:47.029237 14020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:47.045768 14020 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40677
I20260925 11:05:47.045790 14086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40677 every 8 connection(s)
I20260925 11:05:47.047180 14020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:47.050966 14087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:47.056665 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 14020
I20260925 11:05:47.056838 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:47.057253 25215 external_mini_cluster.cc:1428] Setting key db9090f73dd85123c1d86ee54153cb75
I20260925 11:05:47.061419 14087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1: Bootstrap starting.
I20260925 11:05:47.064649 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:47.058962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:39121 (local address 127.0.0.1:40677)
0925 11:05:47.059758 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:47.059771 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:47.059812 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:05:47.061418 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:47.061440 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:47.061803 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:47.062110 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:47.063161 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:47.063167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:47.063180 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:47.063194 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:47.063254 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:47.063293 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:47.063457 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:47.063574 (+ 117us) server_negotiation.cc:299] Negotiation successful
0925 11:05:47.063752 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":366,"threads_started":1}
I20260925 11:05:47.064888 14087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:47.065976 14087 log.cc:824] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:47.068288 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:47.068859 14087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1: No bootstrap required, opened a new log
I20260925 11:05:47.075651 14087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edee7d9dfad4941855191d16e51b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40677 } }
I20260925 11:05:47.076018 14087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:47.076133 14087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5edee7d9dfad4941855191d16e51b3a1, State: Initialized, Role: FOLLOWER
I20260925 11:05:47.076694 14087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edee7d9dfad4941855191d16e51b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40677 } }
I20260925 11:05:47.076915 14087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:47.077037 14087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:47.077181 14087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:47.079706 14087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edee7d9dfad4941855191d16e51b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40677 } }
I20260925 11:05:47.080291 14087 leader_election.cc:302] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5edee7d9dfad4941855191d16e51b3a1; no voters:
I20260925 11:05:47.080516 14090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:47.077781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:33943 (local address 127.0.0.1:40677)
0925 11:05:47.077966 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:47.077972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:47.078738 (+ 766us) server_negotiation.cc:407] Connection header received
0925 11:05:47.078887 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:47.078893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:47.078955 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:47.079054 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:47.079876 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:47.079882 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:47.079886 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:47.079903 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:47.079947 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:47.079965 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:47.080091 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:47.080135 (+ 44us) server_negotiation.cc:299] Negotiation successful
0925 11:05:47.080191 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:05:47.080916 14087 leader_election.cc:288] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:47.081015 14097 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:47.082293 14097 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [term 1 LEADER]: Becoming Leader. State: Replica: 5edee7d9dfad4941855191d16e51b3a1, State: Running, Role: LEADER
I20260925 11:05:47.082788 14097 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edee7d9dfad4941855191d16e51b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40677 } }
I20260925 11:05:47.082998 14087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:47.085086 14099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5edee7d9dfad4941855191d16e51b3a1. Latest consensus state: current_term: 1 leader_uuid: "5edee7d9dfad4941855191d16e51b3a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edee7d9dfad4941855191d16e51b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40677 } } }
I20260925 11:05:47.085376 14099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:47.085891 14098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5edee7d9dfad4941855191d16e51b3a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5edee7d9dfad4941855191d16e51b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40677 } } }
I20260925 11:05:47.086158 14098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:47.086326 14104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:47.097532 14104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:47.099777 14114 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260925 11:05:47.106921 14104 catalog_manager.cc:1347] Generated new cluster ID: 79e888527f014d488e1ce1d44d53e634
I20260925 11:05:47.107018 14104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:47.115401 14104 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:47.115868 14104 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:47.124799 14104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5edee7d9dfad4941855191d16e51b3a1: Generated new TSK 0
I20260925 11:05:47.125931 14104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:47.138661 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14020
W20260925 11:05:47.139730 14082 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.198:39121
W20260925 11:05:47.140413 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14020: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:47.140560 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 14020
2026-09-25T11:05:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1759 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:47 dist-test-slave-trms krb5kdc[14118](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:47 dist-test-slave-trms krb5kdc[14118](info): set up 2 sockets
Sep 25 11:05:47 dist-test-slave-trms krb5kdc[14118](info): commencing operation
krb5kdc: starting...
W20260925 11:05:48.146493 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:48 dist-test-slave-trms krb5kdc[14118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:48.384660 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:48.704463 14131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:48.704746 14131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:48.704873 14131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:48.711835 14131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:48.711956 14131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:48.712034 14131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:48.712118 14131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:48.712177 14131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:48.721644 14131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38523
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:48.722914 14131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:48.724169 14131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:48.735184 14136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:48.735157 14139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:48.735255 14137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:48.735754 14131 server_base.cc:1034] running on GCE node
I20260925 11:05:48.736452 14131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:48.737744 14131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:48.738932 14131 hybrid_clock.cc:648] HybridClock initialized: now 1790334348738885 us; error 36 us; skew 500 ppm
I20260925 11:05:48.739249 14131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:48 dist-test-slave-trms krb5kdc[14118](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334348, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:48.745092 14131 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:48.746532 14131 webserver.cc:460] Webserver started at http://127.0.0.1:43903/ using document root <none> and password file <none>
I20260925 11:05:48.747169 14131 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:48.747285 14131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:48.747565 14131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:48.750519 14131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "e0e0620aa91b4a49a2f69a6656469a81"
format_stamp: "Formatted at 2026-09-25 11:05:48 on dist-test-slave-trms"
server_key: "d51b15857db85230a7bda18193dbef9c"
server_key_iv: "a4b76be13467ff0b0bca2d578af3a024"
server_key_version: "encryptionkey@0"
I20260925 11:05:48.751240 14131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "e0e0620aa91b4a49a2f69a6656469a81"
format_stamp: "Formatted at 2026-09-25 11:05:48 on dist-test-slave-trms"
server_key: "d51b15857db85230a7bda18193dbef9c"
server_key_iv: "a4b76be13467ff0b0bca2d578af3a024"
server_key_version: "encryptionkey@0"
I20260925 11:05:48.756073 14131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:05:48.759299 14146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:48.760535 14131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:05:48.760694 14131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "e0e0620aa91b4a49a2f69a6656469a81"
format_stamp: "Formatted at 2026-09-25 11:05:48 on dist-test-slave-trms"
server_key: "d51b15857db85230a7bda18193dbef9c"
server_key_iv: "a4b76be13467ff0b0bca2d578af3a024"
server_key_version: "encryptionkey@0"
I20260925 11:05:48.760874 14131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:48.787652 14131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:48.791555 14131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:48.791827 14131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:48.808419 14131 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38523
I20260925 11:05:48.808450 14197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38523 every 8 connection(s)
I20260925 11:05:48.809744 14131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:48.813257 14198 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:48.817050 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 14131
I20260925 11:05:48.817224 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:48.817741 25215 external_mini_cluster.cc:1428] Setting key ff313faf5792781a8d978babb9f1c5b6
I20260925 11:05:48.825672 14198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81: Bootstrap starting.
I20260925 11:05:48.829339 14198 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:48.830616 14198 log.cc:824] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81: Log is configured to *not* fsync() on all Append() calls
Sep 25 11:05:48 dist-test-slave-trms krb5kdc[14118](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260925 11:05:48.833431 14198 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81: No bootstrap required, opened a new log
I20260925 11:05:48.839545 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:48.819479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:47281 (local address 127.0.0.1:38523)
0925 11:05:48.820244 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:48.820256 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:48.820297 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:05:48.821844 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:48.821867 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:48.822346 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:48.822670 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:48.824293 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:48.825563 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:48.827247 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:48.827904 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:48.828024 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:48.832532 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:48.832540 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:48.832554 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:48.832588 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:48.835470 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:48.836212 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:48.836218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:48.836224 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:48.836390 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:48.836825 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:48.836829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:48.836831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:48.837337 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:48.837652 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:48.838129 (+ 477us) server_negotiation.cc:299] Negotiation successful
0925 11:05:48.838519 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":359,"threads_started":1}
I20260925 11:05:48.841338 14198 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e0620aa91b4a49a2f69a6656469a81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38523 } }
I20260925 11:05:48.841683 14198 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:48.841784 14198 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e0620aa91b4a49a2f69a6656469a81, State: Initialized, Role: FOLLOWER
I20260925 11:05:48.842427 14198 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e0620aa91b4a49a2f69a6656469a81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38523 } }
I20260925 11:05:48.842660 14198 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:48.842784 14198 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:48.842948 14198 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:48.843403 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:48.845872 14198 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e0620aa91b4a49a2f69a6656469a81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38523 } }
I20260925 11:05:48.846467 14198 leader_election.cc:302] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0e0620aa91b4a49a2f69a6656469a81; no voters:
I20260925 11:05:48.847106 14198 leader_election.cc:288] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:48.847221 14205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:48.849426 14198 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:48.849915 14205 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [term 1 LEADER]: Becoming Leader. State: Replica: e0e0620aa91b4a49a2f69a6656469a81, State: Running, Role: LEADER
I20260925 11:05:48.850495 14205 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e0620aa91b4a49a2f69a6656469a81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38523 } }
I20260925 11:05:48.854684 14209 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0e0620aa91b4a49a2f69a6656469a81. Latest consensus state: current_term: 1 leader_uuid: "e0e0620aa91b4a49a2f69a6656469a81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e0620aa91b4a49a2f69a6656469a81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38523 } } }
I20260925 11:05:48.854719 14208 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0e0620aa91b4a49a2f69a6656469a81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0e0620aa91b4a49a2f69a6656469a81" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38523 } } }
I20260925 11:05:48.855082 14208 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:48.855078 14209 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:48.857494 14215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:48.863018 14215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:48.869293 14201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:48.849287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:51525 (local address 127.0.0.1:38523)
0925 11:05:48.849464 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:48.849470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:48.849493 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:05:48.849648 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:48.849653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:48.849742 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:48.849815 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:48.851873 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:48.853100 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:48.857026 (+ 3926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:48.857692 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:48.857824 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:48.861027 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:48.861034 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:48.861038 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:48.861077 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:48.865059 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:48.866154 (+ 1095us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:48.866159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:48.866163 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:48.866334 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:48.866897 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:48.866902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:48.866905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:48.867330 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:48.867592 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:48.868762 (+ 1170us) server_negotiation.cc:299] Negotiation successful
0925 11:05:48.868947 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260925 11:05:48.872944 14215 catalog_manager.cc:1347] Generated new cluster ID: 566a67d24c0f4c43b788bbed61f8cbaa
I20260925 11:05:48.873116 14215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:48.892786 14215 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:48.894698 14215 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:48.908346 14215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0e0620aa91b4a49a2f69a6656469a81: Generated new TSK 0
I20260925 11:05:48.909449 14215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:48.917410 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14131
I20260925 11:05:48.918467 14193 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:49.007422 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 14131
W20260925 11:05:49.021122 14116 connection.cc:535] client connection to 127.0.0.1:38523 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38523): Connection reset by peer (error 104)
2026-09-25T11:05:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1866 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:49 dist-test-slave-trms krb5kdc[14230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:49 dist-test-slave-trms krb5kdc[14230](info): set up 2 sockets
Sep 25 11:05:49 dist-test-slave-trms krb5kdc[14230](info): commencing operation
krb5kdc: starting...
W20260925 11:05:50.010583 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:50 dist-test-slave-trms krb5kdc[14230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:50.250739 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43235
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:50.566083 14243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:50.566401 14243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:50.566531 14243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:50.573711 14243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:50.573825 14243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:50.573901 14243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:50.573982 14243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:50.574039 14243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:50.583841 14243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43235
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:50.585140 14243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:50.586483 14243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:50.597982 14249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:50.598109 14251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:50.598075 14248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:50.599078 14243 server_base.cc:1034] running on GCE node
I20260925 11:05:50.599792 14243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:50.601167 14243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:50.602375 14243 hybrid_clock.cc:648] HybridClock initialized: now 1790334350602320 us; error 52 us; skew 500 ppm
I20260925 11:05:50.602686 14243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:50 dist-test-slave-trms krb5kdc[14230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334350, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:50.605651 14243 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:50.607138 14243 webserver.cc:460] Webserver started at http://127.0.0.1:41649/ using document root <none> and password file <none>
I20260925 11:05:50.607789 14243 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:50.607903 14243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:50.608183 14243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:50.611128 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "a980ccb178fa43ab8c37dfcb4a639517"
format_stamp: "Formatted at 2026-09-25 11:05:50 on dist-test-slave-trms"
server_key: "f9013094f949212d3866bf9cf6ad8b13"
server_key_iv: "d7ec78f14d2c45442d56b6a134888872"
server_key_version: "encryptionkey@0"
I20260925 11:05:50.611898 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "a980ccb178fa43ab8c37dfcb4a639517"
format_stamp: "Formatted at 2026-09-25 11:05:50 on dist-test-slave-trms"
server_key: "f9013094f949212d3866bf9cf6ad8b13"
server_key_iv: "d7ec78f14d2c45442d56b6a134888872"
server_key_version: "encryptionkey@0"
I20260925 11:05:50.616914 14243 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:05:50.620255 14258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:50.621630 14243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:05:50.621791 14243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "a980ccb178fa43ab8c37dfcb4a639517"
format_stamp: "Formatted at 2026-09-25 11:05:50 on dist-test-slave-trms"
server_key: "f9013094f949212d3866bf9cf6ad8b13"
server_key_iv: "d7ec78f14d2c45442d56b6a134888872"
server_key_version: "encryptionkey@0"
I20260925 11:05:50.621961 14243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:50.644012 14243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:50.648211 14243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:50.648473 14243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:50.664836 14243 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43235
I20260925 11:05:50.664868 14309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43235 every 8 connection(s)
I20260925 11:05:50.666103 14243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:50.669669 14310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:50.672523 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 14243
I20260925 11:05:50.672675 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:50.673084 25215 external_mini_cluster.cc:1428] Setting key d32b1abed3630b07124c95b6dc87a139
I20260925 11:05:50.682374 14310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517: Bootstrap starting.
I20260925 11:05:50.686158 14310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:50.687541 14310 log.cc:824] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:50.687824 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:50.674902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:33453 (local address 127.0.0.1:43235)
0925 11:05:50.675638 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:50.675654 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:50.675703 (+ 49us) server_negotiation.cc:407] Connection header received
0925 11:05:50.677917 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:50.677948 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:50.678544 (+ 596us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:50.678990 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:50.679527 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:50.681478 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:50.682798 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:50.683768 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:50.683985 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:50.685026 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:50.685036 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:50.685053 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:50.685773 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:50.685863 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:50.685895 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:50.686062 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:50.686298 (+ 236us) server_negotiation.cc:299] Negotiation successful
0925 11:05:50.686493 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":287,"threads_started":1}
I20260925 11:05:50.690496 14310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517: No bootstrap required, opened a new log
I20260925 11:05:50.691601 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:50.698293 14310 raft_consensus.cc:348] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a980ccb178fa43ab8c37dfcb4a639517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43235 } }
I20260925 11:05:50.698670 14310 raft_consensus.cc:374] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:50.698757 14310 raft_consensus.cc:729] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a980ccb178fa43ab8c37dfcb4a639517, State: Initialized, Role: FOLLOWER
I20260925 11:05:50.699394 14310 consensus_queue.cc:260] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a980ccb178fa43ab8c37dfcb4a639517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43235 } }
I20260925 11:05:50.699689 14310 raft_consensus.cc:388] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:50.699834 14310 raft_consensus.cc:482] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:50.700038 14310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:50.701669 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:50.695627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:36739 (local address 127.0.0.1:43235)
0925 11:05:50.695784 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:50.695791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:50.696188 (+ 397us) server_negotiation.cc:407] Connection header received
0925 11:05:50.696361 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:50.696365 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:50.696430 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:50.696515 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:50.696996 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:50.697726 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:50.699150 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:50.699786 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:50.699919 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:50.700964 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:50.700972 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:50.700977 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:50.700997 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:50.701046 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:50.701066 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:50.701177 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:50.701279 (+ 102us) server_negotiation.cc:299] Negotiation successful
0925 11:05:50.701336 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260925 11:05:50.702844 14310 raft_consensus.cc:504] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a980ccb178fa43ab8c37dfcb4a639517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43235 } }
I20260925 11:05:50.703317 14310 leader_election.cc:302] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a980ccb178fa43ab8c37dfcb4a639517; no voters:
I20260925 11:05:50.703817 14310 leader_election.cc:288] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:50.703938 14320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:50.704576 14320 raft_consensus.cc:686] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [term 1 LEADER]: Becoming Leader. State: Replica: a980ccb178fa43ab8c37dfcb4a639517, State: Running, Role: LEADER
I20260925 11:05:50.705078 14320 consensus_queue.cc:237] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a980ccb178fa43ab8c37dfcb4a639517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43235 } }
I20260925 11:05:50.705662 14310 sys_catalog.cc:564] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:50.707701 14321 sys_catalog.cc:455] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a980ccb178fa43ab8c37dfcb4a639517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a980ccb178fa43ab8c37dfcb4a639517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43235 } } }
I20260925 11:05:50.707800 14322 sys_catalog.cc:455] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a980ccb178fa43ab8c37dfcb4a639517. Latest consensus state: current_term: 1 leader_uuid: "a980ccb178fa43ab8c37dfcb4a639517" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a980ccb178fa43ab8c37dfcb4a639517" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43235 } } }
I20260925 11:05:50.708058 14321 sys_catalog.cc:458] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:50.708134 14322 sys_catalog.cc:458] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:50.708882 14328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:50.714811 14328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:50.724315 14328 catalog_manager.cc:1347] Generated new cluster ID: 848bd75954b846ffa7258bae74ee9a7c
I20260925 11:05:50.724438 14328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:50.761193 14328 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:50.763098 14328 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:50.771324 14328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a980ccb178fa43ab8c37dfcb4a639517: Generated new TSK 0
I20260925 11:05:50.772333 14328 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:50.782814 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14243
W20260925 11:05:50.784322 14305 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.198:33453
W20260925 11:05:50.785110 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14243: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:50.785298 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 14243
W20260925 11:05:50.800230 14228 connection.cc:535] client connection to 127.0.0.1:43235 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43235): Connection reset by peer (error 104)
2026-09-25T11:05:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1779 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:50 dist-test-slave-trms krb5kdc[14341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:50 dist-test-slave-trms krb5kdc[14341](info): set up 2 sockets
Sep 25 11:05:50 dist-test-slave-trms krb5kdc[14341](info): commencing operation
krb5kdc: starting...
W20260925 11:05:51.794762 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.004s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:51 dist-test-slave-trms krb5kdc[14341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260925 11:05:52.029311 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41615
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:52.348876 14354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:52.349196 14354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:52.349342 14354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:52.356513 14354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:52.356630 14354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:52.356710 14354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:52.356793 14354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:52.356853 14354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:52.366688 14354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41615
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:52.367997 14354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:52.369349 14354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:52.380360 14359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:52.380442 14360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:52.380403 14362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:52.381168 14354 server_base.cc:1034] running on GCE node
I20260925 11:05:52.381933 14354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:52.383354 14354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:52.384563 14354 hybrid_clock.cc:648] HybridClock initialized: now 1790334352384523 us; error 31 us; skew 500 ppm
I20260925 11:05:52.384905 14354 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:52 dist-test-slave-trms krb5kdc[14341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334352, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:52.388072 14354 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260925 11:05:52.389510 14354 webserver.cc:460] Webserver started at http://127.0.0.1:33443/ using document root <none> and password file <none>
I20260925 11:05:52.390164 14354 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:52.390329 14354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:52.390651 14354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:52.393680 14354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "05e6f4f4db9a40d486e48d4a5255aacd"
format_stamp: "Formatted at 2026-09-25 11:05:52 on dist-test-slave-trms"
server_key: "8cd088927c3b5ae56ea0c6a1860173de"
server_key_iv: "774266e04a48c2a8bd001a5db8ef73fe"
server_key_version: "encryptionkey@0"
I20260925 11:05:52.394495 14354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "05e6f4f4db9a40d486e48d4a5255aacd"
format_stamp: "Formatted at 2026-09-25 11:05:52 on dist-test-slave-trms"
server_key: "8cd088927c3b5ae56ea0c6a1860173de"
server_key_iv: "774266e04a48c2a8bd001a5db8ef73fe"
server_key_version: "encryptionkey@0"
I20260925 11:05:52.399636 14354 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260925 11:05:52.403043 14369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:52.404496 14354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:05:52.404659 14354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "05e6f4f4db9a40d486e48d4a5255aacd"
format_stamp: "Formatted at 2026-09-25 11:05:52 on dist-test-slave-trms"
server_key: "8cd088927c3b5ae56ea0c6a1860173de"
server_key_iv: "774266e04a48c2a8bd001a5db8ef73fe"
server_key_version: "encryptionkey@0"
I20260925 11:05:52.404848 14354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:52.410470 14354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:52.414772 14354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:52.415045 14354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:52.431972 14354 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41615
I20260925 11:05:52.432008 14420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41615 every 8 connection(s)
I20260925 11:05:52.433420 14354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:52.437372 14421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:52.442912 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 14354
I20260925 11:05:52.443074 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:52.443476 25215 external_mini_cluster.cc:1428] Setting key a6faa2b8561170cf448aec8bac2b59f4
I20260925 11:05:52.447865 14421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd: Bootstrap starting.
I20260925 11:05:52.451225 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:52.445313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:32975 (local address 127.0.0.1:41615)
0925 11:05:52.446138 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:52.446150 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:52.446239 (+ 89us) server_negotiation.cc:407] Connection header received
0925 11:05:52.447901 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:52.447925 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:52.448294 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:52.448613 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:52.449667 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:52.449674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:52.449688 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:52.449702 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:52.449767 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:52.449787 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:52.449972 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:52.450066 (+ 94us) server_negotiation.cc:299] Negotiation successful
0925 11:05:52.450284 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":343,"threads_started":1}
I20260925 11:05:52.451350 14421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:52.452678 14421 log.cc:824] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:52.455241 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:52.455736 14421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd: No bootstrap required, opened a new log
I20260925 11:05:52.461503 14424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:52.459223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.198:36347 (local address 127.0.0.1:41615)
0925 11:05:52.459411 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:52.459417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:52.459746 (+ 329us) server_negotiation.cc:407] Connection header received
0925 11:05:52.459901 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:52.459907 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:52.459959 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:52.460039 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:52.460905 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:52.460912 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:52.460916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:52.460930 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:52.460965 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:52.460983 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:52.461075 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:52.461194 (+ 119us) server_negotiation.cc:299] Negotiation successful
0925 11:05:52.461242 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260925 11:05:52.464046 14421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e6f4f4db9a40d486e48d4a5255aacd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } }
I20260925 11:05:52.464416 14421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:52.464510 14421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05e6f4f4db9a40d486e48d4a5255aacd, State: Initialized, Role: FOLLOWER
I20260925 11:05:52.465060 14421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e6f4f4db9a40d486e48d4a5255aacd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } }
I20260925 11:05:52.465281 14421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:52.465411 14421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:52.465606 14421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:52.467840 14421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e6f4f4db9a40d486e48d4a5255aacd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } }
I20260925 11:05:52.468339 14421 leader_election.cc:302] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05e6f4f4db9a40d486e48d4a5255aacd; no voters:
I20260925 11:05:52.468958 14421 leader_election.cc:288] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:52.469089 14431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:52.469741 14431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [term 1 LEADER]: Becoming Leader. State: Replica: 05e6f4f4db9a40d486e48d4a5255aacd, State: Running, Role: LEADER
I20260925 11:05:52.470283 14431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e6f4f4db9a40d486e48d4a5255aacd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } }
I20260925 11:05:52.471035 14421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:52.473403 14432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05e6f4f4db9a40d486e48d4a5255aacd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e6f4f4db9a40d486e48d4a5255aacd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } } }
I20260925 11:05:52.473769 14432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:52.474635 14433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05e6f4f4db9a40d486e48d4a5255aacd. Latest consensus state: current_term: 1 leader_uuid: "05e6f4f4db9a40d486e48d4a5255aacd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e6f4f4db9a40d486e48d4a5255aacd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41615 } } }
I20260925 11:05:52.474978 14433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:52.476176 14436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:52.481326 14436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:52.489521 14436 catalog_manager.cc:1347] Generated new cluster ID: b99cab242335447ca5930a970bec321f
I20260925 11:05:52.489640 14436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:52.515599 14436 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:52.516034 14436 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:52.527645 14436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05e6f4f4db9a40d486e48d4a5255aacd: Generated new TSK 0
I20260925 11:05:52.528487 14436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:52.539408 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14354
W20260925 11:05:52.540508 14416 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.198:32975
W20260925 11:05:52.541234 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14354: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:52.541369 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 14354
2026-09-25T11:05:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1755 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:52 dist-test-slave-trms krb5kdc[14452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:52 dist-test-slave-trms krb5kdc[14452](info): set up 2 sockets
Sep 25 11:05:52 dist-test-slave-trms krb5kdc[14452](info): commencing operation
krb5kdc: starting...
W20260925 11:05:53.558833 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:53 dist-test-slave-trms krb5kdc[14452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:05:53.794297 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43931
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:43931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:54.113932 14465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:54.114265 14465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:54.114415 14465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:54.121657 14465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:54.121783 14465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:54.121860 14465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:54.121943 14465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:54.122004 14465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:54.132210 14465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:43931
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:43931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:54.133518 14465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:54.134851 14465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:54.146013 14470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:54.146013 14473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:54.146049 14471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:54.146865 14465 server_base.cc:1034] running on GCE node
I20260925 11:05:54.147601 14465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:54.148972 14465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:54.150236 14465 hybrid_clock.cc:648] HybridClock initialized: now 1790334354150143 us; error 50 us; skew 500 ppm
I20260925 11:05:54.150574 14465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:54 dist-test-slave-trms krb5kdc[14452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334354, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:54.156910 14465 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:05:54.158352 14465 webserver.cc:460] Webserver started at http://127.24.159.254:39633/ using document root <none> and password file <none>
I20260925 11:05:54.159013 14465 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:54.159137 14465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:54.159512 14465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:54.163698 14465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "39e0035b03fa4386a740a86b6df0a27a"
format_stamp: "Formatted at 2026-09-25 11:05:54 on dist-test-slave-trms"
server_key: "cd3e027d3d89fcd8169f33d8dae5bc80"
server_key_iv: "0357728e4d67ccfcfdc02589f5369ea8"
server_key_version: "encryptionkey@0"
I20260925 11:05:54.164723 14465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "39e0035b03fa4386a740a86b6df0a27a"
format_stamp: "Formatted at 2026-09-25 11:05:54 on dist-test-slave-trms"
server_key: "cd3e027d3d89fcd8169f33d8dae5bc80"
server_key_iv: "0357728e4d67ccfcfdc02589f5369ea8"
server_key_version: "encryptionkey@0"
I20260925 11:05:54.171494 14465 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260925 11:05:54.174968 14480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:54.176288 14465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260925 11:05:54.176452 14465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "39e0035b03fa4386a740a86b6df0a27a"
format_stamp: "Formatted at 2026-09-25 11:05:54 on dist-test-slave-trms"
server_key: "cd3e027d3d89fcd8169f33d8dae5bc80"
server_key_iv: "0357728e4d67ccfcfdc02589f5369ea8"
server_key_version: "encryptionkey@0"
I20260925 11:05:54.176630 14465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:54.200475 14465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:54.206099 14465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:54.206411 14465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:54.222977 14465 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:43931
I20260925 11:05:54.223018 14531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:43931 every 8 connection(s)
I20260925 11:05:54.224313 14465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:54.227245 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 14465
I20260925 11:05:54.227419 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:54.227938 25215 external_mini_cluster.cc:1428] Setting key e714285717a3d6f23cb519f2f0cf96aa
I20260925 11:05:54.228298 14532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:54.242291 14532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a: Bootstrap starting.
Sep 25 11:05:54 dist-test-slave-trms krb5kdc[14452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334353, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:54.245906 14532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:54.247033 14532 log.cc:824] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:54.250272 14532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a: No bootstrap required, opened a new log
I20260925 11:05:54.251273 14535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:54.229594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49351 (local address 127.24.159.254:43931)
0925 11:05:54.230354 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:54.230366 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:54.230406 (+ 40us) server_negotiation.cc:407] Connection header received
0925 11:05:54.232754 (+ 2348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:54.232777 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:54.233194 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:54.233543 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:54.235270 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:54.236555 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:54.238310 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:54.239055 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:54.239199 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:54.244027 (+ 4828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:54.244036 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:54.244053 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:54.244102 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:54.247089 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:54.248033 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:54.248041 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:54.248052 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:54.248266 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:54.248734 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:54.248737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:54.248739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:54.249258 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:54.249486 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:54.249949 (+ 463us) server_negotiation.cc:299] Negotiation successful
0925 11:05:54.250250 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":335,"threads_started":1}
I20260925 11:05:54.255133 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:54.258610 14532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0035b03fa4386a740a86b6df0a27a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43931 } }
I20260925 11:05:54.259018 14532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:54.259151 14532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39e0035b03fa4386a740a86b6df0a27a, State: Initialized, Role: FOLLOWER
I20260925 11:05:54.259865 14532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0035b03fa4386a740a86b6df0a27a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43931 } }
I20260925 11:05:54.260093 14532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:54.260219 14532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:54.260372 14532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:54.262604 14532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0035b03fa4386a740a86b6df0a27a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43931 } }
I20260925 11:05:54.263057 14532 leader_election.cc:302] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39e0035b03fa4386a740a86b6df0a27a; no voters:
I20260925 11:05:54.263619 14532 leader_election.cc:288] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:54.263751 14542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:54.264864 14542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [term 1 LEADER]: Becoming Leader. State: Replica: 39e0035b03fa4386a740a86b6df0a27a, State: Running, Role: LEADER
I20260925 11:05:54.265390 14542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0035b03fa4386a740a86b6df0a27a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43931 } }
I20260925 11:05:54.265854 14532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:54.267827 14544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39e0035b03fa4386a740a86b6df0a27a. Latest consensus state: current_term: 1 leader_uuid: "39e0035b03fa4386a740a86b6df0a27a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0035b03fa4386a740a86b6df0a27a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43931 } } }
I20260925 11:05:54.268142 14544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:54.268684 14543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39e0035b03fa4386a740a86b6df0a27a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39e0035b03fa4386a740a86b6df0a27a" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 43931 } } }
I20260925 11:05:54.268988 14543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:54.269582 14549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:54.276250 14535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:54.259258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34305 (local address 127.24.159.254:43931)
0925 11:05:54.259419 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:54.259425 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:54.259761 (+ 336us) server_negotiation.cc:407] Connection header received
0925 11:05:54.259919 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:54.259924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:54.259988 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:54.260065 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:54.261456 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:54.262280 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:54.263626 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:54.264327 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:54.265255 (+ 928us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:54.267696 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:54.267703 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:54.267708 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:54.267755 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:54.271693 (+ 3938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:54.272773 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:54.272778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:54.272782 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:54.272960 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:54.273523 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:54.273528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:54.273532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:54.273969 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:54.275478 (+ 1509us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:54.275654 (+ 176us) server_negotiation.cc:299] Negotiation successful
0925 11:05:54.275859 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260925 11:05:54.288568 14549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:54.297012 14549 catalog_manager.cc:1347] Generated new cluster ID: 787de5e83df042e2951332c085fa170c
I20260925 11:05:54.297122 14549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:54.305421 14549 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:54.306825 14549 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:54.312467 14549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39e0035b03fa4386a740a86b6df0a27a: Generated new TSK 0
I20260925 11:05:54.313529 14549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:54.327986 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14465
I20260925 11:05:54.329527 14527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:54.420161 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 14465
W20260925 11:05:54.433743 14450 connection.cc:535] client connection to 127.24.159.254:43931 recv error: Network error: failed to read from TLS socket (remote: 127.24.159.254:43931): Connection reset by peer (error 104)
2026-09-25T11:05:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1878 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:54 dist-test-slave-trms krb5kdc[14564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:54 dist-test-slave-trms krb5kdc[14564](info): set up 2 sockets
Sep 25 11:05:54 dist-test-slave-trms krb5kdc[14564](info): commencing operation
krb5kdc: starting...
W20260925 11:05:55.430589 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.978s user 0.002s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:55 dist-test-slave-trms krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:05:55.673504 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36711
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:36711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:55.994068 14577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:55.994377 14577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:55.994477 14577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:56.001639 14577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:56.001758 14577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:56.001811 14577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:56.001868 14577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:56.001909 14577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:56.011497 14577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:36711
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:36711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:56.012854 14577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:56.014142 14577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:56.025434 14585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:56.025488 14583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:56.026150 14577 server_base.cc:1034] running on GCE node
W20260925 11:05:56.026351 14582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:56.027441 14577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:56.029309 14577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:56.030534 14577 hybrid_clock.cc:648] HybridClock initialized: now 1790334356030480 us; error 43 us; skew 500 ppm
I20260925 11:05:56.030896 14577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:56 dist-test-slave-trms krb5kdc[14564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334356, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:56.034349 14577 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:05:56.035751 14577 webserver.cc:460] Webserver started at http://127.24.159.254:46191/ using document root <none> and password file <none>
I20260925 11:05:56.036410 14577 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:56.036527 14577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:56.036798 14577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:56.039786 14577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "0b032fadb0b047de8d4913a66c66be64"
format_stamp: "Formatted at 2026-09-25 11:05:56 on dist-test-slave-trms"
server_key: "beadeba7ab7f227d39509ccb78f838a8"
server_key_iv: "1deb87763b50ef616049c660214d8fce"
server_key_version: "encryptionkey@0"
I20260925 11:05:56.040530 14577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "0b032fadb0b047de8d4913a66c66be64"
format_stamp: "Formatted at 2026-09-25 11:05:56 on dist-test-slave-trms"
server_key: "beadeba7ab7f227d39509ccb78f838a8"
server_key_iv: "1deb87763b50ef616049c660214d8fce"
server_key_version: "encryptionkey@0"
I20260925 11:05:56.045493 14577 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:05:56.048743 14592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:56.050042 14577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:05:56.050240 14577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "0b032fadb0b047de8d4913a66c66be64"
format_stamp: "Formatted at 2026-09-25 11:05:56 on dist-test-slave-trms"
server_key: "beadeba7ab7f227d39509ccb78f838a8"
server_key_iv: "1deb87763b50ef616049c660214d8fce"
server_key_version: "encryptionkey@0"
I20260925 11:05:56.050426 14577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:56.065066 14577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:56.068890 14577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:56.069156 14577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:56.085716 14577 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:36711
I20260925 11:05:56.085733 14643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:36711 every 8 connection(s)
I20260925 11:05:56.087101 14577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:56.090759 14644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:56.096418 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 14577
I20260925 11:05:56.096606 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:56.097129 25215 external_mini_cluster.cc:1428] Setting key 9487c18d81550857137ab6e152d21282
I20260925 11:05:56.101778 14644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64: Bootstrap starting.
I20260925 11:05:56.104835 14644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:56.105809 14644 log.cc:824] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:56.108542 14644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64: No bootstrap required, opened a new log
I20260925 11:05:56.110348 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:56.099105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48485 (local address 127.24.159.254:36711)
0925 11:05:56.099939 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:56.099950 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:56.099991 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:05:56.101653 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:56.101677 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:56.102121 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:56.102531 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:56.103060 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:56.104318 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:56.105655 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:56.106434 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:56.106617 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:56.107779 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:56.107788 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:56.107803 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:56.108317 (+ 514us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:56.108388 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:56.108416 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:56.108555 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:56.108846 (+ 291us) server_negotiation.cc:299] Negotiation successful
0925 11:05:56.109057 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":358,"threads_started":1}
I20260925 11:05:56.113999 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:56.117591 14644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b032fadb0b047de8d4913a66c66be64" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36711 } }
I20260925 11:05:56.118057 14644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:56.118167 14644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b032fadb0b047de8d4913a66c66be64, State: Initialized, Role: FOLLOWER
I20260925 11:05:56.118798 14644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b032fadb0b047de8d4913a66c66be64" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36711 } }
I20260925 11:05:56.119055 14644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:56.119190 14644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:56.119359 14644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:56.122125 14644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b032fadb0b047de8d4913a66c66be64" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36711 } }
I20260925 11:05:56.122663 14644 leader_election.cc:302] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b032fadb0b047de8d4913a66c66be64; no voters:
I20260925 11:05:56.123291 14644 leader_election.cc:288] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:56.123390 14654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:56.124832 14654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [term 1 LEADER]: Becoming Leader. State: Replica: 0b032fadb0b047de8d4913a66c66be64, State: Running, Role: LEADER
I20260925 11:05:56.125614 14644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:56.125649 14654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b032fadb0b047de8d4913a66c66be64" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36711 } }
I20260925 11:05:56.125896 14647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:56.118979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33671 (local address 127.24.159.254:36711)
0925 11:05:56.119239 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:56.119245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:56.119506 (+ 261us) server_negotiation.cc:407] Connection header received
0925 11:05:56.119673 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:56.119678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:56.119752 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:56.119878 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:56.120277 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:56.121367 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:56.122714 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:56.123401 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:56.123522 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:56.124844 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:56.124851 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:56.124857 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:56.124891 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:56.124960 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:56.124986 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:56.125073 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:56.125388 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 11:05:56.125460 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260925 11:05:56.131459 14655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b032fadb0b047de8d4913a66c66be64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b032fadb0b047de8d4913a66c66be64" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36711 } } }
I20260925 11:05:56.132675 14655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:56.131635 14656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b032fadb0b047de8d4913a66c66be64. Latest consensus state: current_term: 1 leader_uuid: "0b032fadb0b047de8d4913a66c66be64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b032fadb0b047de8d4913a66c66be64" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 36711 } } }
I20260925 11:05:56.133422 14656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:56.133641 14670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:56.137504 14670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:56.146116 14670 catalog_manager.cc:1347] Generated new cluster ID: 8a61d7e6c5e944a6aa9bb47a02ca8bf3
I20260925 11:05:56.146240 14670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:56.157927 14670 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:56.159271 14670 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:56.171576 14670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b032fadb0b047de8d4913a66c66be64: Generated new TSK 0
I20260925 11:05:56.172816 14670 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:56.187669 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14577
W20260925 11:05:56.188834 14639 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48485
W20260925 11:05:56.189553 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14577: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:56.189673 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 14577
W20260925 11:05:56.203377 14562 connection.cc:535] client connection to 127.24.159.254:36711 recv error: Network error: failed to read from TLS socket (remote: 127.24.159.254:36711): Connection reset by peer (error 104)
2026-09-25T11:05:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1769 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:56 dist-test-slave-trms krb5kdc[14675](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:56 dist-test-slave-trms krb5kdc[14675](info): set up 2 sockets
Sep 25 11:05:56 dist-test-slave-trms krb5kdc[14675](info): commencing operation
krb5kdc: starting...
W20260925 11:05:57.210572 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:57 dist-test-slave-trms krb5kdc[14675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:05:57.451476 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45355
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:40369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:45355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:57.771854 14688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:57.772135 14688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:57.772269 14688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:57.779351 14688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:57.779465 14688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:57.779538 14688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:57.779613 14688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:57.779675 14688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:57.789384 14688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:40369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:45355
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:57.790683 14688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:57.791986 14688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:57.803062 14694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:57.803028 14693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:57.803094 14696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:57.803942 14688 server_base.cc:1034] running on GCE node
I20260925 11:05:57.804728 14688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:57.806115 14688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:57.807385 14688 hybrid_clock.cc:648] HybridClock initialized: now 1790334357807323 us; error 52 us; skew 500 ppm
I20260925 11:05:57.807735 14688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:57 dist-test-slave-trms krb5kdc[14675](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334357, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:57.810649 14688 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:05:57.812036 14688 webserver.cc:460] Webserver started at http://127.24.159.254:43517/ using document root <none> and password file <none>
I20260925 11:05:57.812664 14688 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:57.812760 14688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:57.812992 14688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:57.815837 14688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "f45ce970ac654a16b4a5301b339400e7"
format_stamp: "Formatted at 2026-09-25 11:05:57 on dist-test-slave-trms"
server_key: "1aeda48ddcd87935093467374b6ed5b2"
server_key_iv: "208ac1ccae26bba23a95872a356aae24"
server_key_version: "encryptionkey@0"
I20260925 11:05:57.816527 14688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "f45ce970ac654a16b4a5301b339400e7"
format_stamp: "Formatted at 2026-09-25 11:05:57 on dist-test-slave-trms"
server_key: "1aeda48ddcd87935093467374b6ed5b2"
server_key_iv: "208ac1ccae26bba23a95872a356aae24"
server_key_version: "encryptionkey@0"
I20260925 11:05:57.821461 14688 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:05:57.824781 14703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:57.826017 14688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:05:57.826169 14688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "f45ce970ac654a16b4a5301b339400e7"
format_stamp: "Formatted at 2026-09-25 11:05:57 on dist-test-slave-trms"
server_key: "1aeda48ddcd87935093467374b6ed5b2"
server_key_iv: "208ac1ccae26bba23a95872a356aae24"
server_key_version: "encryptionkey@0"
I20260925 11:05:57.826383 14688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:57.831763 14688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:57.835418 14688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:57.835672 14688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:57.852304 14688 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:45355
I20260925 11:05:57.852325 14754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:45355 every 8 connection(s)
I20260925 11:05:57.853667 14688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:57.857372 14755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:05:57.864009 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 14688
I20260925 11:05:57.864164 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:57.864565 25215 external_mini_cluster.cc:1428] Setting key 30c78ea7f6f2531f231e4d1d6144ff98
I20260925 11:05:57.867892 14755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7: Bootstrap starting.
I20260925 11:05:57.871373 14755 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:57.872224 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:57.866490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40715 (local address 127.24.159.254:45355)
0925 11:05:57.867255 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:57.867268 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:57.867315 (+ 47us) server_negotiation.cc:407] Connection header received
0925 11:05:57.868968 (+ 1653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:57.868991 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:57.869381 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:57.869700 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:57.870731 (+ 1031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:57.870737 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:57.870751 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:57.870764 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:57.870822 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:57.870842 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:57.870973 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:57.871117 (+ 144us) server_negotiation.cc:299] Negotiation successful
0925 11:05:57.871287 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":325,"threads_started":1}
I20260925 11:05:57.872464 14755 log.cc:824] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:57.875427 14755 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7: No bootstrap required, opened a new log
I20260925 11:05:57.875779 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:57.882094 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:57.879746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58049 (local address 127.24.159.254:45355)
0925 11:05:57.879915 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:57.879921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:57.880384 (+ 463us) server_negotiation.cc:407] Connection header received
0925 11:05:57.880543 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:57.880547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:57.880597 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:57.880671 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:57.881513 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:57.881517 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:57.881520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:05:57.881534 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:57.881569 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:05:57.881589 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:57.881684 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:57.881793 (+ 109us) server_negotiation.cc:299] Negotiation successful
0925 11:05:57.881837 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:05:57.883672 14755 raft_consensus.cc:348] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ce970ac654a16b4a5301b339400e7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45355 } }
I20260925 11:05:57.884037 14755 raft_consensus.cc:374] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:57.884109 14755 raft_consensus.cc:729] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f45ce970ac654a16b4a5301b339400e7, State: Initialized, Role: FOLLOWER
I20260925 11:05:57.884624 14755 consensus_queue.cc:260] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ce970ac654a16b4a5301b339400e7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45355 } }
I20260925 11:05:57.884833 14755 raft_consensus.cc:388] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:57.884939 14755 raft_consensus.cc:482] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:57.885066 14755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:57.887181 14755 raft_consensus.cc:504] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ce970ac654a16b4a5301b339400e7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45355 } }
I20260925 11:05:57.887635 14755 leader_election.cc:302] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f45ce970ac654a16b4a5301b339400e7; no voters:
I20260925 11:05:57.888216 14755 leader_election.cc:288] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:57.888320 14765 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:57.889048 14765 raft_consensus.cc:686] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [term 1 LEADER]: Becoming Leader. State: Replica: f45ce970ac654a16b4a5301b339400e7, State: Running, Role: LEADER
I20260925 11:05:57.889523 14765 consensus_queue.cc:237] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ce970ac654a16b4a5301b339400e7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45355 } }
I20260925 11:05:57.890270 14755 sys_catalog.cc:564] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:57.892102 14767 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f45ce970ac654a16b4a5301b339400e7. Latest consensus state: current_term: 1 leader_uuid: "f45ce970ac654a16b4a5301b339400e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ce970ac654a16b4a5301b339400e7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45355 } } }
I20260925 11:05:57.892408 14767 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:57.893015 14766 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f45ce970ac654a16b4a5301b339400e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f45ce970ac654a16b4a5301b339400e7" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45355 } } }
I20260925 11:05:57.893201 14770 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:57.893293 14766 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:57.902254 14770 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:57.910970 14770 catalog_manager.cc:1347] Generated new cluster ID: b2eb2338b9df497b88bad41da51630e5
I20260925 11:05:57.911082 14770 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:57.931025 14770 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:57.931533 14770 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:57.943135 14770 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f45ce970ac654a16b4a5301b339400e7: Generated new TSK 0
I20260925 11:05:57.944228 14770 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:57.957779 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14688
W20260925 11:05:57.958942 14750 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40715
W20260925 11:05:57.959649 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14688: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:05:57.959777 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 14688
2026-09-25T11:05:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1769 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:58 dist-test-slave-trms krb5kdc[14786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:58 dist-test-slave-trms krb5kdc[14786](info): set up 2 sockets
Sep 25 11:05:58 dist-test-slave-trms krb5kdc[14786](info): commencing operation
krb5kdc: starting...
W20260925 11:05:58.970623 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.008s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:05:59 dist-test-slave-trms krb5kdc[14786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:05:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:05:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:05:59.199162 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:35507
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:42785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:35507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:05:59.515336 14799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:05:59.515638 14799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:05:59.515779 14799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:05:59.522861 14799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:05:59.522989 14799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:05:59.523069 14799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:05:59.523146 14799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:05:59.523206 14799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:05:59.534065 14799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:42785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:35507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:35507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:05:59.535362 14799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:05:59.536628 14799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:05:59.547405 14804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:59.547379 14805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:05:59.547370 14807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:05:59.547992 14799 server_base.cc:1034] running on GCE node
I20260925 11:05:59.548832 14799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:05:59.550251 14799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:05:59.551471 14799 hybrid_clock.cc:648] HybridClock initialized: now 1790334359551427 us; error 38 us; skew 500 ppm
I20260925 11:05:59.551770 14799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:05:59 dist-test-slave-trms krb5kdc[14786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334359, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:05:59.558013 14799 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:05:59.559583 14799 webserver.cc:460] Webserver started at http://127.24.159.254:35087/ using document root <none> and password file <none>
I20260925 11:05:59.560266 14799 fs_manager.cc:359] Metadata directory not provided
I20260925 11:05:59.560388 14799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:05:59.560669 14799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:05:59.563633 14799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "e3198f23eab846f59db8a56d54e8c4b6"
format_stamp: "Formatted at 2026-09-25 11:05:59 on dist-test-slave-trms"
server_key: "cac9573b10150cf049ba3ac3468ca0ab"
server_key_iv: "f94170f2b553bca81e17b77127fb58cb"
server_key_version: "encryptionkey@0"
I20260925 11:05:59.564347 14799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "e3198f23eab846f59db8a56d54e8c4b6"
format_stamp: "Formatted at 2026-09-25 11:05:59 on dist-test-slave-trms"
server_key: "cac9573b10150cf049ba3ac3468ca0ab"
server_key_iv: "f94170f2b553bca81e17b77127fb58cb"
server_key_version: "encryptionkey@0"
I20260925 11:05:59.569214 14799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260925 11:05:59.572779 14814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:05:59.574038 14799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:05:59.574182 14799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "e3198f23eab846f59db8a56d54e8c4b6"
format_stamp: "Formatted at 2026-09-25 11:05:59 on dist-test-slave-trms"
server_key: "cac9573b10150cf049ba3ac3468ca0ab"
server_key_iv: "f94170f2b553bca81e17b77127fb58cb"
server_key_version: "encryptionkey@0"
I20260925 11:05:59.574406 14799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:05:59.608121 14799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:05:59.612325 14799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:05:59.612574 14799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:05:59.629127 14799 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:35507
I20260925 11:05:59.629163 14865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:35507 every 8 connection(s)
I20260925 11:05:59.630548 14799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:05:59.631433 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 14799
I20260925 11:05:59.631621 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:05:59.632149 25215 external_mini_cluster.cc:1428] Setting key e0e37d113a3f26da639010e96ca68a81
I20260925 11:05:59.634984 14866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 25 11:05:59 dist-test-slave-trms krb5kdc[14786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:05:59.647661 14866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6: Bootstrap starting.
I20260925 11:05:59.651690 14866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6: Neither blocks nor log segments found. Creating new log.
I20260925 11:05:59.652980 14866 log.cc:824] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6: Log is configured to *not* fsync() on all Append() calls
I20260925 11:05:59.655133 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:59.633913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52903 (local address 127.24.159.254:35507)
0925 11:05:59.634661 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:59.634673 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:59.634712 (+ 39us) server_negotiation.cc:407] Connection header received
0925 11:05:59.636336 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:59.636359 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:59.636731 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:59.637033 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:59.638731 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:59.640050 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:59.641775 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:59.642448 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:59.642568 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:59.647085 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:59.647096 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:59.647113 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:59.647163 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:59.650870 (+ 3707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:59.651523 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:59.651532 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:59.651541 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:59.651746 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:59.652130 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:59.652138 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:59.652142 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:59.652838 (+ 696us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:59.653128 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:59.653443 (+ 315us) server_negotiation.cc:299] Negotiation successful
0925 11:05:59.653917 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":335,"threads_started":1}
I20260925 11:05:59.656215 14866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6: No bootstrap required, opened a new log
I20260925 11:05:59.658600 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:05:59.663800 14866 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3198f23eab846f59db8a56d54e8c4b6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35507 } }
I20260925 11:05:59.664151 14866 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:05:59.664242 14866 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3198f23eab846f59db8a56d54e8c4b6, State: Initialized, Role: FOLLOWER
I20260925 11:05:59.664834 14866 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3198f23eab846f59db8a56d54e8c4b6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35507 } }
I20260925 11:05:59.665134 14866 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:05:59.665277 14866 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:05:59.665450 14866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:05:59.668136 14866 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3198f23eab846f59db8a56d54e8c4b6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35507 } }
I20260925 11:05:59.668674 14866 leader_election.cc:302] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3198f23eab846f59db8a56d54e8c4b6; no voters:
I20260925 11:05:59.669358 14866 leader_election.cc:288] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:05:59.669440 14876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:05:59.670614 14876 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [term 1 LEADER]: Becoming Leader. State: Replica: e3198f23eab846f59db8a56d54e8c4b6, State: Running, Role: LEADER
I20260925 11:05:59.671144 14876 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3198f23eab846f59db8a56d54e8c4b6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35507 } }
I20260925 11:05:59.671674 14866 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:05:59.674033 14877 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3198f23eab846f59db8a56d54e8c4b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3198f23eab846f59db8a56d54e8c4b6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35507 } } }
I20260925 11:05:59.674009 14878 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3198f23eab846f59db8a56d54e8c4b6. Latest consensus state: current_term: 1 leader_uuid: "e3198f23eab846f59db8a56d54e8c4b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3198f23eab846f59db8a56d54e8c4b6" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 35507 } } }
I20260925 11:05:59.674429 14878 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:59.674429 14877 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6 [sys.catalog]: This master's current role is: LEADER
I20260925 11:05:59.679247 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:05:59.662483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46707 (local address 127.24.159.254:35507)
0925 11:05:59.662666 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 11:05:59.662672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:05:59.663136 (+ 464us) server_negotiation.cc:407] Connection header received
0925 11:05:59.663257 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:05:59.663262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:05:59.663334 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:05:59.663445 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:05:59.664811 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:59.665876 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:59.667073 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:05:59.667701 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:05:59.667816 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:05:59.670676 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:05:59.670683 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:05:59.670690 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:05:59.670732 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:05:59.674471 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:59.675363 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:59.675369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:59.675386 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:59.675571 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:05:59.676049 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:05:59.676055 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:05:59.676059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:05:59.676579 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:05:59.676917 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0925 11:05:59.678562 (+ 1645us) server_negotiation.cc:299] Negotiation successful
0925 11:05:59.678852 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260925 11:05:59.687142 14881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:05:59.691648 14881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:05:59.700139 14881 catalog_manager.cc:1347] Generated new cluster ID: 23e958409cb44a64bfb4d5134470bb0f
I20260925 11:05:59.700258 14881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:05:59.713967 14881 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:05:59.715842 14881 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:05:59.726224 14881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3198f23eab846f59db8a56d54e8c4b6: Generated new TSK 0
I20260925 11:05:59.727281 14881 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:05:59.733738 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14799
I20260925 11:05:59.735095 14861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:05:59.822563 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 14799
2026-09-25T11:05:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1862 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:05:59 dist-test-slave-trms krb5kdc[14898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:05:59 dist-test-slave-trms krb5kdc[14898](info): set up 2 sockets
Sep 25 11:05:59 dist-test-slave-trms krb5kdc[14898](info): commencing operation
krb5kdc: starting...
W20260925 11:06:00.842546 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:06:00 dist-test-slave-trms krb5kdc[14898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:06:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:06:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:06:01.083518 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:39211
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:39211
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:06:01.405413 14911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:06:01.405702 14911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:06:01.405834 14911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:06:01.413146 14911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:06:01.413277 14911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:06:01.413354 14911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:06:01.413432 14911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:06:01.413499 14911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:06:01.423584 14911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:39211
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:39211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:06:01.424904 14911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:06:01.426273 14911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:06:01.437479 14916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:01.437559 14919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:01.437534 14917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:06:01.438244 14911 server_base.cc:1034] running on GCE node
I20260925 11:06:01.439007 14911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:06:01.440412 14911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:06:01.441624 14911 hybrid_clock.cc:648] HybridClock initialized: now 1790334361441578 us; error 37 us; skew 500 ppm
I20260925 11:06:01.441969 14911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:06:01 dist-test-slave-trms krb5kdc[14898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:06:01.445004 14911 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:06:01.446498 14911 webserver.cc:460] Webserver started at http://127.24.159.254:40135/ using document root <none> and password file <none>
I20260925 11:06:01.447153 14911 fs_manager.cc:359] Metadata directory not provided
I20260925 11:06:01.447269 14911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:06:01.447561 14911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:06:01.450596 14911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "4ee8757905c04ef3848483b4d80b1748"
format_stamp: "Formatted at 2026-09-25 11:06:01 on dist-test-slave-trms"
server_key: "6a8bb91297535cd42de2adece9fedb7f"
server_key_iv: "46c7f88a3b204eec42ffac582189d5f7"
server_key_version: "encryptionkey@0"
I20260925 11:06:01.451344 14911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "4ee8757905c04ef3848483b4d80b1748"
format_stamp: "Formatted at 2026-09-25 11:06:01 on dist-test-slave-trms"
server_key: "6a8bb91297535cd42de2adece9fedb7f"
server_key_iv: "46c7f88a3b204eec42ffac582189d5f7"
server_key_version: "encryptionkey@0"
I20260925 11:06:01.456297 14911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:06:01.459638 14926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:06:01.460881 14911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260925 11:06:01.461046 14911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "4ee8757905c04ef3848483b4d80b1748"
format_stamp: "Formatted at 2026-09-25 11:06:01 on dist-test-slave-trms"
server_key: "6a8bb91297535cd42de2adece9fedb7f"
server_key_iv: "46c7f88a3b204eec42ffac582189d5f7"
server_key_version: "encryptionkey@0"
I20260925 11:06:01.461220 14911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:06:01.481786 14911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:06:01.485495 14911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:06:01.485762 14911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:06:01.502559 14911 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:39211
I20260925 11:06:01.502594 14977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:39211 every 8 connection(s)
I20260925 11:06:01.503857 14911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:06:01.505868 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 14911
I20260925 11:06:01.506052 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:06:01.506613 25215 external_mini_cluster.cc:1428] Setting key 40a19338bd7976fe07c887c6c3d4f155
I20260925 11:06:01.508019 14978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:06:01.519402 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:01.508378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46107 (local address 127.24.159.254:39211)
0925 11:06:01.509107 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:01.509119 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:01.509173 (+ 54us) server_negotiation.cc:407] Connection header received
0925 11:06:01.510767 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:01.510789 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:01.511193 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:01.511525 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:01.512021 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:01.513311 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:01.515058 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:01.515759 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:01.515892 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:01.517111 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:01.517118 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:01.517132 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:06:01.517690 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:01.517770 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:06:01.517802 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:01.517912 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:01.518111 (+ 199us) server_negotiation.cc:299] Negotiation successful
0925 11:06:01.518423 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":327,"threads_started":1}
I20260925 11:06:01.520435 14978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748: Bootstrap starting.
I20260925 11:06:01.522883 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:06:01.523641 14978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748: Neither blocks nor log segments found. Creating new log.
I20260925 11:06:01.524809 14978 log.cc:824] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748: Log is configured to *not* fsync() on all Append() calls
I20260925 11:06:01.527642 14978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748: No bootstrap required, opened a new log
I20260925 11:06:01.533771 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:01.527575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56353 (local address 127.24.159.254:39211)
0925 11:06:01.527767 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:01.527773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:01.528270 (+ 497us) server_negotiation.cc:407] Connection header received
0925 11:06:01.528467 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:01.528471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:01.528531 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:01.528602 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:01.529162 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:01.530069 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:01.531463 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:01.531924 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:01.532028 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:01.533085 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:01.533090 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:01.533093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:06:01.533110 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:01.533149 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:06:01.533163 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:01.533240 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:01.533382 (+ 142us) server_negotiation.cc:299] Negotiation successful
0925 11:06:01.533468 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:06:01.536537 14978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee8757905c04ef3848483b4d80b1748" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39211 } }
I20260925 11:06:01.537024 14978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:06:01.537137 14978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ee8757905c04ef3848483b4d80b1748, State: Initialized, Role: FOLLOWER
I20260925 11:06:01.537729 14978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee8757905c04ef3848483b4d80b1748" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39211 } }
I20260925 11:06:01.537993 14978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:06:01.538174 14978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:06:01.538409 14978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:06:01.541206 14978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee8757905c04ef3848483b4d80b1748" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39211 } }
I20260925 11:06:01.541747 14978 leader_election.cc:302] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ee8757905c04ef3848483b4d80b1748; no voters:
I20260925 11:06:01.542344 14978 leader_election.cc:288] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:06:01.542445 14988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:06:01.543071 14988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [term 1 LEADER]: Becoming Leader. State: Replica: 4ee8757905c04ef3848483b4d80b1748, State: Running, Role: LEADER
I20260925 11:06:01.543658 14988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee8757905c04ef3848483b4d80b1748" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39211 } }
I20260925 11:06:01.544406 14978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:06:01.546078 14990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ee8757905c04ef3848483b4d80b1748. Latest consensus state: current_term: 1 leader_uuid: "4ee8757905c04ef3848483b4d80b1748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee8757905c04ef3848483b4d80b1748" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39211 } } }
I20260925 11:06:01.546435 14990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [sys.catalog]: This master's current role is: LEADER
I20260925 11:06:01.546876 14989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ee8757905c04ef3848483b4d80b1748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ee8757905c04ef3848483b4d80b1748" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39211 } } }
I20260925 11:06:01.547168 14989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748 [sys.catalog]: This master's current role is: LEADER
I20260925 11:06:01.547511 14993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:06:01.553541 14993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:06:01.562464 14993 catalog_manager.cc:1347] Generated new cluster ID: 781adee9ddc64583ae4d18f45d3a6833
I20260925 11:06:01.562574 14993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:06:01.573062 14993 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:06:01.574350 14993 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:06:01.582376 14993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ee8757905c04ef3848483b4d80b1748: Generated new TSK 0
I20260925 11:06:01.583199 14993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:06:01.596694 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14911
W20260925 11:06:01.597755 14973 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46107
W20260925 11:06:01.598445 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 14911: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:06:01.598582 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 14911
2026-09-25T11:06:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1775 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:06:01 dist-test-slave-trms krb5kdc[15009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:06:01 dist-test-slave-trms krb5kdc[15009](info): set up 2 sockets
Sep 25 11:06:01 dist-test-slave-trms krb5kdc[15009](info): commencing operation
krb5kdc: starting...
W20260925 11:06:02.610586 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.006s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:06:02 dist-test-slave-trms krb5kdc[15009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:06:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:06:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:06:02.859243 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:41903
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:39319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:41903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:06:03.178413 15022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:06:03.178716 15022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:06:03.178860 15022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:06:03.185877 15022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:06:03.185997 15022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:06:03.186077 15022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:06:03.186156 15022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:06:03.186234 15022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:06:03.195894 15022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:39319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:41903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:41903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:06:03.197176 15022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:06:03.198520 15022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:06:03.209493 15027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:03.209566 15028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:03.209605 15030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:06:03.210703 15022 server_base.cc:1034] running on GCE node
I20260925 11:06:03.211467 15022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:06:03.212790 15022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:06:03.213981 15022 hybrid_clock.cc:648] HybridClock initialized: now 1790334363213939 us; error 35 us; skew 500 ppm
I20260925 11:06:03.214344 15022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:06:03 dist-test-slave-trms krb5kdc[15009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334363, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:06:03.217186 15022 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:06:03.218562 15022 webserver.cc:460] Webserver started at http://127.24.159.254:33237/ using document root <none> and password file <none>
I20260925 11:06:03.219209 15022 fs_manager.cc:359] Metadata directory not provided
I20260925 11:06:03.219321 15022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:06:03.219597 15022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:06:03.222532 15022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "812f66cdf70c44eb8fbfbe25312fe28f"
format_stamp: "Formatted at 2026-09-25 11:06:03 on dist-test-slave-trms"
server_key: "1f513eb4964e540251ccd7dd6fc6be36"
server_key_iv: "d85572d418d25a20cce8f9cdb6914cca"
server_key_version: "encryptionkey@0"
I20260925 11:06:03.223263 15022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "812f66cdf70c44eb8fbfbe25312fe28f"
format_stamp: "Formatted at 2026-09-25 11:06:03 on dist-test-slave-trms"
server_key: "1f513eb4964e540251ccd7dd6fc6be36"
server_key_iv: "d85572d418d25a20cce8f9cdb6914cca"
server_key_version: "encryptionkey@0"
I20260925 11:06:03.228161 15022 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:06:03.231534 15037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:06:03.232771 15022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:06:03.232931 15022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "812f66cdf70c44eb8fbfbe25312fe28f"
format_stamp: "Formatted at 2026-09-25 11:06:03 on dist-test-slave-trms"
server_key: "1f513eb4964e540251ccd7dd6fc6be36"
server_key_iv: "d85572d418d25a20cce8f9cdb6914cca"
server_key_version: "encryptionkey@0"
I20260925 11:06:03.233107 15022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:06:03.238437 15022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:06:03.242121 15022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:06:03.242398 15022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:06:03.258472 15022 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:41903
I20260925 11:06:03.258490 15088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:41903 every 8 connection(s)
I20260925 11:06:03.259776 15022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:06:03.261237 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 15022
I20260925 11:06:03.261399 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:06:03.261930 25215 external_mini_cluster.cc:1428] Setting key 357b149ebc647e287be6fdf745ec941c
I20260925 11:06:03.264002 15089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:06:03.269531 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:03.263776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49749 (local address 127.24.159.254:41903)
0925 11:06:03.264580 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:03.264592 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:03.264632 (+ 40us) server_negotiation.cc:407] Connection header received
0925 11:06:03.266224 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:03.266252 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:03.266602 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:03.266978 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:03.267964 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:03.267970 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:03.267983 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:06:03.267997 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:03.268073 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:06:03.268092 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:03.268319 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:03.268457 (+ 138us) server_negotiation.cc:299] Negotiation successful
0925 11:06:03.268629 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":364,"threads_started":1}
I20260925 11:06:03.273276 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:06:03.275345 15089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f: Bootstrap starting.
I20260925 11:06:03.278757 15089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f: Neither blocks nor log segments found. Creating new log.
I20260925 11:06:03.279726 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:03.277419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40421 (local address 127.24.159.254:41903)
0925 11:06:03.277604 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:03.277610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:03.277938 (+ 328us) server_negotiation.cc:407] Connection header received
0925 11:06:03.278104 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:03.278107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:03.278152 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:03.278267 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:03.279146 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:03.279150 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:03.279156 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0925 11:06:03.279171 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:03.279211 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0925 11:06:03.279230 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:03.279313 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:03.279431 (+ 118us) server_negotiation.cc:299] Negotiation successful
0925 11:06:03.279475 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 11:06:03.280050 15089 log.cc:824] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f: Log is configured to *not* fsync() on all Append() calls
I20260925 11:06:03.282742 15089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f: No bootstrap required, opened a new log
I20260925 11:06:03.289862 15089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f66cdf70c44eb8fbfbe25312fe28f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41903 } }
I20260925 11:06:03.290264 15089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260925 11:06:03.290349 15089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 812f66cdf70c44eb8fbfbe25312fe28f, State: Initialized, Role: FOLLOWER
I20260925 11:06:03.290866 15089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f66cdf70c44eb8fbfbe25312fe28f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41903 } }
I20260925 11:06:03.291074 15089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260925 11:06:03.291203 15089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260925 11:06:03.291358 15089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:06:03.293432 15089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f66cdf70c44eb8fbfbe25312fe28f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41903 } }
I20260925 11:06:03.293893 15089 leader_election.cc:302] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 812f66cdf70c44eb8fbfbe25312fe28f; no voters:
I20260925 11:06:03.294456 15089 leader_election.cc:288] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260925 11:06:03.294576 15099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:06:03.295825 15099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [term 1 LEADER]: Becoming Leader. State: Replica: 812f66cdf70c44eb8fbfbe25312fe28f, State: Running, Role: LEADER
I20260925 11:06:03.296360 15099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f66cdf70c44eb8fbfbe25312fe28f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41903 } }
I20260925 11:06:03.296527 15089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:06:03.299062 15101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 812f66cdf70c44eb8fbfbe25312fe28f. Latest consensus state: current_term: 1 leader_uuid: "812f66cdf70c44eb8fbfbe25312fe28f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f66cdf70c44eb8fbfbe25312fe28f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41903 } } }
I20260925 11:06:03.299503 15101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [sys.catalog]: This master's current role is: LEADER
I20260925 11:06:03.299124 15100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "812f66cdf70c44eb8fbfbe25312fe28f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "812f66cdf70c44eb8fbfbe25312fe28f" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 41903 } } }
I20260925 11:06:03.299865 15100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f [sys.catalog]: This master's current role is: LEADER
I20260925 11:06:03.300446 15107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:06:03.306403 15107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:06:03.315321 15107 catalog_manager.cc:1347] Generated new cluster ID: fa701f96d8dd45b39b847198b73fcd92
I20260925 11:06:03.315430 15107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:06:03.328263 15107 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:06:03.328667 15107 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:06:03.343557 15107 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 812f66cdf70c44eb8fbfbe25312fe28f: Generated new TSK 0
I20260925 11:06:03.344367 15107 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:06:03.359372 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 15022
W20260925 11:06:03.360442 15084 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49749
W20260925 11:06:03.361075 25215 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 15022: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260925 11:06:03.361192 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 15022
2026-09-25T11:06:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1762 ms)
[----------] 18 tests from AuthTokenIssuingTest (32435 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:06:03 dist-test-slave-trms krb5kdc[15120](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:06:03 dist-test-slave-trms krb5kdc[15120](info): set up 2 sockets
Sep 25 11:06:03 dist-test-slave-trms krb5kdc[15120](info): commencing operation
krb5kdc: starting...
W20260925 11:06:04.374689 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.005s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:06:04 dist-test-slave-trms krb5kdc[15120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:06:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:06:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:06:04.623227 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:39913
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:39913,127.24.159.253:45507,127.24.159.252:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:06:04.936061 15133 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:06:04.936420 15133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:06:04.936514 15133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:06:04.936626 15133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:06:04.943784 15133 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:06:04.943881 15133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:06:04.943959 15133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:06:04.944026 15133 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:06:04.944108 15133 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:06:04.953785 15133 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:39913,127.24.159.253:45507,127.24.159.252:38651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:39913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:06:04.955044 15133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:06:04.956331 15133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:06:04.967250 15138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:04.967407 15139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:04.967517 15141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:06:04.968319 15133 server_base.cc:1034] running on GCE node
I20260925 11:06:04.969053 15133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:06:04.970502 15133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:06:04.971741 15133 hybrid_clock.cc:648] HybridClock initialized: now 1790334364971664 us; error 59 us; skew 500 ppm
I20260925 11:06:04.972083 15133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:06:04 dist-test-slave-trms krb5kdc[15120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334364, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:06:04.978844 15133 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:06:04.980366 15133 webserver.cc:460] Webserver started at http://127.24.159.254:45811/ using document root <none> and password file <none>
I20260925 11:06:04.981074 15133 fs_manager.cc:359] Metadata directory not provided
I20260925 11:06:04.981172 15133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:06:04.981472 15133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:06:04.984386 15133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "806c8074e9b7436b93abc3319c76f451"
format_stamp: "Formatted at 2026-09-25 11:06:04 on dist-test-slave-trms"
server_key: "b1692a0e1c9b67cc4db314c2a7e1911a"
server_key_iv: "738d0172ab665031ab7901091440a849"
server_key_version: "encryptionkey@0"
I20260925 11:06:04.985128 15133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "806c8074e9b7436b93abc3319c76f451"
format_stamp: "Formatted at 2026-09-25 11:06:04 on dist-test-slave-trms"
server_key: "b1692a0e1c9b67cc4db314c2a7e1911a"
server_key_iv: "738d0172ab665031ab7901091440a849"
server_key_version: "encryptionkey@0"
I20260925 11:06:04.989993 15133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260925 11:06:04.993199 15148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:06:04.994495 15133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260925 11:06:04.994640 15133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "806c8074e9b7436b93abc3319c76f451"
format_stamp: "Formatted at 2026-09-25 11:06:04 on dist-test-slave-trms"
server_key: "b1692a0e1c9b67cc4db314c2a7e1911a"
server_key_iv: "738d0172ab665031ab7901091440a849"
server_key_version: "encryptionkey@0"
I20260925 11:06:04.994827 15133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:06:05.016106 15133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:06:05.019990 15133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:06:05.020228 15133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:06:05.036664 15133 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:39913
I20260925 11:06:05.036689 15199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:39913 every 8 connection(s)
I20260925 11:06:05.037995 15133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:06:05.041616 15200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:06:05.045683 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 15133
I20260925 11:06:05.045903 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:06:05.046684 25215 external_mini_cluster.cc:1428] Setting key 9b43002436b14de667993ee88dcbbb30
I20260925 11:06:05.049225 15200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:06:05 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334364, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:06:05.071280 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.052728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:39913 (local address 127.0.0.1:52674)
0925 11:06:05.053552 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.053592 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:05.055304 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:05.056188 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:05.056193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:05.056742 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:05.057055 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:05.057071 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.058747 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.058753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:05.060299 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:05.060318 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.061198 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.061204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:05.061354 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:05.061951 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:05.061981 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:05.064308 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:05.067278 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:05.067289 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:05.067300 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:05.067679 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:05.068059 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:05.068063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:05.068067 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:05.068206 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:05.068955 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:05.068963 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:05.069175 (+ 212us) client_negotiation.cc:769] Sending connection context
0925 11:06:05.069398 (+ 223us) client_negotiation.cc:240] Negotiation successful
0925 11:06:05.069987 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":171,"thread_start_us":333,"threads_started":1}
I20260925 11:06:05.071312 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.052728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52674 (local address 127.24.159.254:39913)
0925 11:06:05.053522 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:05.053535 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:05.053708 (+ 173us) server_negotiation.cc:407] Connection header received
0925 11:06:05.055680 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:05.055687 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:05.055969 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:05.056184 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:05.057260 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.058600 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.060434 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.060977 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.061097 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:05.064508 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:05.064515 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:05.064527 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:05.064557 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:05.067163 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:05.067793 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:05.067801 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:05.067805 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:05.067959 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:05.068302 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:05.068307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:05.068311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:05.068829 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:05.069026 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:05.069507 (+ 481us) server_negotiation.cc:299] Negotiation successful
0925 11:06:05.069858 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":614,"spinlock_wait_cycles":11648,"thread_start_us":410,"threads_started":1}
WARNING: no policy specified for kudu/127.24.159.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.253@KRBTEST.COM" created.
I20260925 11:06:05.074293 15200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:06:05.075338 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.074952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43776)
0925 11:06:05.075120 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.075223 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260925 11:06:05.075640 15149 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
I20260925 11:06:05.077068 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.076750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43786)
0925 11:06:05.076898 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.076962 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260925 11:06:05.077258 15200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:45507: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab.
Entry for principal kudu/127.24.159.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab.
I20260925 11:06:05.101250 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.253:45507
--webserver_interface=127.24.159.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab
--principal=kudu/127.24.159.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:39913,127.24.159.253:45507,127.24.159.252:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:06:05.126571 15200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } attempt: 1
I20260925 11:06:05.127657 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.127212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43798)
0925 11:06:05.127402 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.127521 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 11:06:05.128654 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.128330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43806)
0925 11:06:05.128460 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.128529 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 11:06:05.128820 15200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:45507: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
I20260925 11:06:05.197031 15200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } attempt: 2
I20260925 11:06:05.198182 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.197717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43822)
0925 11:06:05.197956 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.198052 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260925 11:06:05.199160 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.198812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43838)
0925 11:06:05.198960 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.199040 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260925 11:06:05.199349 15200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:45507: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
I20260925 11:06:05.290566 15200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } attempt: 3
I20260925 11:06:05.291568 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.291160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43852)
0925 11:06:05.291344 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.291433 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 11:06:05.292539 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.292212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43862)
0925 11:06:05.292348 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.292414 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 11:06:05.292737 15200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:45507: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
W20260925 11:06:05.417899 15207 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:06:05.418277 15207 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:06:05.418376 15207 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:06:05.418504 15207 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:06:05.425807 15207 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:06:05.425921 15207 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:06:05.426010 15207 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:06:05.426079 15207 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:06:05.426163 15207 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:06:05.435953 15200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } attempt: 4
I20260925 11:06:05.436199 15207 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:39913,127.24.159.253:45507,127.24.159.252:38651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab
--principal=kudu/127.24.159.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.253:45507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.159.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:06:05.437121 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.436604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43870)
0925 11:06:05.436903 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.436990 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260925 11:06:05.437790 15207 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:06:05.438033 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.437721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43878)
0925 11:06:05.437833 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.437908 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260925 11:06:05.438221 15200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:45507: Network error: Client connection negotiation failed: client connection to 127.24.159.253:45507: connect: Connection refused (error 111)
I20260925 11:06:05.439229 15207 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:06:05.450240 15213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:05.450273 15215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:05.450307 15212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:06:05.451437 15207 server_base.cc:1034] running on GCE node
I20260925 11:06:05.452174 15207 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:06:05.453562 15207 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:06:05.454766 15207 hybrid_clock.cc:648] HybridClock initialized: now 1790334365454701 us; error 36 us; skew 500 ppm
I20260925 11:06:05.455076 15207 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:06:05 dist-test-slave-trms krb5kdc[15120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334365, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:06:05.458063 15207 init.cc:377] Logged in from keytab as kudu/127.24.159.253@KRBTEST.COM (short username kudu)
I20260925 11:06:05.459486 15207 webserver.cc:460] Webserver started at http://127.24.159.253:45435/ using document root <none> and password file <none>
I20260925 11:06:05.460141 15207 fs_manager.cc:359] Metadata directory not provided
I20260925 11:06:05.460239 15207 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:06:05.460522 15207 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:06:05.463433 15207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/data/instance:
uuid: "d035093a7c68457abc39c1da5823afc1"
format_stamp: "Formatted at 2026-09-25 11:06:05 on dist-test-slave-trms"
server_key: "bd921df468cc30f5a1476c7fd8f15a46"
server_key_iv: "9c9c01006fe0f7990f5cb87b1a7d6ff5"
server_key_version: "encryptionkey@0"
I20260925 11:06:05.464172 15207 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/wal/instance:
uuid: "d035093a7c68457abc39c1da5823afc1"
format_stamp: "Formatted at 2026-09-25 11:06:05 on dist-test-slave-trms"
server_key: "bd921df468cc30f5a1476c7fd8f15a46"
server_key_iv: "9c9c01006fe0f7990f5cb87b1a7d6ff5"
server_key_version: "encryptionkey@0"
I20260925 11:06:05.469009 15207 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260925 11:06:05.472244 15222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:06:05.473510 15207 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:06:05.473668 15207 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/wal
uuid: "d035093a7c68457abc39c1da5823afc1"
format_stamp: "Formatted at 2026-09-25 11:06:05 on dist-test-slave-trms"
server_key: "bd921df468cc30f5a1476c7fd8f15a46"
server_key_iv: "9c9c01006fe0f7990f5cb87b1a7d6ff5"
server_key_version: "encryptionkey@0"
I20260925 11:06:05.473852 15207 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:06:05.500962 15207 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:06:05.505199 15207 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:06:05.505442 15207 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:06:05.522243 15207 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.253:45507
I20260925 11:06:05.522269 15273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.253:45507 every 8 connection(s)
I20260925 11:06:05.523572 15207 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
I20260925 11:06:05.524080 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 15207
I20260925 11:06:05.524315 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-1/wal/instance
I20260925 11:06:05.524833 25215 external_mini_cluster.cc:1428] Setting key 97b837de42e61adf8b6d4655f2db706c
I20260925 11:06:05.527450 15274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:06:05.533701 15274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.159.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.252@KRBTEST.COM" created.
Sep 25 11:06:05 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334365, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:06:05.555572 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.536294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52686 (local address 127.24.159.254:39913)
0925 11:06:05.536462 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:05.536467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:05.537816 (+ 1349us) server_negotiation.cc:407] Connection header received
0925 11:06:05.539644 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:05.539650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:05.539729 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:05.539815 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:05.541304 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.541969 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.543487 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.543893 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.544001 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:05.549292 (+ 5291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:05.549297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:05.549300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:05.549336 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:05.552321 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:05.553234 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:05.553240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:05.553243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:05.553365 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:05.553774 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:05.553779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:05.553783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:05.554106 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:05.554282 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:05.555031 (+ 749us) server_negotiation.cc:299] Negotiation successful
0925 11:06:05.555258 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260925 11:06:05.556509 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.536918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:39913 (local address 127.0.0.1:52686)
0925 11:06:05.537658 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.537695 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:05.539382 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:05.540055 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:05.540063 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:05.540570 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:05.541139 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:05.541156 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.542120 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.542124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:05.543344 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:05.543355 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.544100 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.544107 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:05.544323 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:05.544978 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:05.545000 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:05.549057 (+ 4057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:05.552459 (+ 3402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:05.552468 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:05.552487 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:05.553107 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:05.553471 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:05.553476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:05.553480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:05.553657 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:05.554255 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:05.554263 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:05.554638 (+ 375us) client_negotiation.cc:769] Sending connection context
0925 11:06:05.554878 (+ 240us) client_negotiation.cc:240] Negotiation successful
0925 11:06:05.555413 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":308,"threads_started":1}
I20260925 11:06:05.558146 15274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:06:05 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334365, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
I20260925 11:06:05.571713 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.558929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43884)
0925 11:06:05.559089 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.559106 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:05.559181 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:05.560110 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:05.560114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:05.560495 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:05.560620 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:05.560628 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.561937 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.561942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:05.562928 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:05.562940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.563680 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.563685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:05.563852 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:05.564357 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:05.564373 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:05.566441 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:05.569360 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:05.569365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:05.569369 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:05.569704 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:05.570037 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:05.570042 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:05.570046 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:05.570176 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:05.570777 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:05.570782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:05.570915 (+ 133us) client_negotiation.cc:769] Sending connection context
0925 11:06:05.571028 (+ 113us) client_negotiation.cc:240] Negotiation successful
0925 11:06:05.571290 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260925 11:06:05.571785 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.559144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43884 (local address 127.24.159.253:45507)
0925 11:06:05.559728 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:05.559734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:05.559757 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:06:05.559797 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:05.559802 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:05.559978 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:05.560109 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:05.560772 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.561809 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.563044 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.563552 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.563660 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:05.566623 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:05.566634 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:05.566647 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:05.566676 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:05.569240 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:05.569819 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:05.569825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:05.569827 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:05.569941 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:05.570282 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:05.570286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:05.570290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:05.570660 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:05.570852 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:05.571194 (+ 342us) server_negotiation.cc:299] Negotiation successful
0925 11:06:05.571412 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":320,"threads_started":1}
I20260925 11:06:05.574039 15274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:06:05.575100 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.574658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38082)
0925 11:06:05.574825 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.574956 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260925 11:06:05.575335 15223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
I20260925 11:06:05.576689 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.576359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38086)
0925 11:06:05.576497 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.576589 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab.
Entry for principal kudu/127.24.159.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab.
W20260925 11:06:05.576860 15274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:38651: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
I20260925 11:06:05.577716 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.252:38651
--webserver_interface=127.24.159.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab
--principal=kudu/127.24.159.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:44699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:39913,127.24.159.253:45507,127.24.159.252:38651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:06:05.626148 15274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } attempt: 1
I20260925 11:06:05.627362 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.626928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38102)
0925 11:06:05.627110 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.627215 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 11:06:05.628482 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.628145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38112)
0925 11:06:05.628306 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.628386 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260925 11:06:05.628686 15274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:38651: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
I20260925 11:06:05.696877 15274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } attempt: 2
I20260925 11:06:05.698006 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.697538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38116)
0925 11:06:05.697720 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.697813 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260925 11:06:05.698931 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.698627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38118)
0925 11:06:05.698758 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.698825 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 11:06:05.699162 15274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:38651: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
I20260925 11:06:05.737439 15200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } attempt: 5
Sep 25 11:06:05 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334364, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
I20260925 11:06:05.749733 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.738146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43888)
0925 11:06:05.738359 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.738380 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:05.738522 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:05.738874 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:05.738880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:05.739253 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:05.739405 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:05.739413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.740361 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.740366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:05.741244 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:05.741256 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.741970 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.741976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:05.742172 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:05.742774 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:05.742792 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:05.744710 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:05.747496 (+ 2786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:05.747502 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:05.747507 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:05.747851 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:05.748183 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:05.748188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:05.748193 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:05.748329 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:05.748813 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:05.748817 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:05.748960 (+ 143us) client_negotiation.cc:769] Sending connection context
0925 11:06:05.749074 (+ 114us) client_negotiation.cc:240] Negotiation successful
0925 11:06:05.749384 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260925 11:06:05.749734 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.738181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43888 (local address 127.24.159.253:45507)
0925 11:06:05.738384 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:05.738389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:05.738458 (+ 69us) server_negotiation.cc:407] Connection header received
0925 11:06:05.738643 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:05.738647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:05.738725 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:05.738810 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:05.739556 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.740236 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.741399 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:05.741832 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:05.741941 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:05.744892 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:05.744899 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:05.744905 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:05.744941 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:05.747372 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:05.747962 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:05.747967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:05.747969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:05.748084 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:05.748423 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:05.748428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:05.748431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:05.748706 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:05.748833 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:05.749201 (+ 368us) server_negotiation.cc:299] Negotiation successful
0925 11:06:05.749423 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 11:06:05.750963 15200 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:06:05.751837 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.751500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38126)
0925 11:06:05.751647 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.751716 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260925 11:06:05.752635 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.752331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38130)
0925 11:06:05.752474 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.752536 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260925 11:06:05.752804 15200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:38651: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
I20260925 11:06:05.790352 15274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } attempt: 3
I20260925 11:06:05.791463 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.791033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38136)
0925 11:06:05.791208 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.791298 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 11:06:05.792443 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.792037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38140)
0925 11:06:05.792242 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.792319 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260925 11:06:05.792631 15274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:38651: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
I20260925 11:06:05.803984 15200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } attempt: 1
I20260925 11:06:05.804881 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.804489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38142)
0925 11:06:05.804659 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.804745 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 11:06:05.805721 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.805417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38146)
0925 11:06:05.805551 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.805616 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 11:06:05.805897 15200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:38651: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
I20260925 11:06:05.874112 15200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } attempt: 2
I20260925 11:06:05.875200 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.874725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38150)
0925 11:06:05.874952 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.875041 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 11:06:05.876106 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.875780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38154)
0925 11:06:05.875938 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.876003 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260925 11:06:05.876291 15200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:38651: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
W20260925 11:06:05.895920 15281 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:06:05.896263 15281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:06:05.896359 15281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:06:05.896478 15281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:06:05.903810 15281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:06:05.903906 15281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:06:05.903995 15281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:06:05.904059 15281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:06:05.904145 15281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:06:05.913969 15281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:44699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:39913,127.24.159.253:45507,127.24.159.252:38651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab
--principal=kudu/127.24.159.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.252:38651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.159.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:06:05.915308 15281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:06:05.916613 15281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:06:05.927819 15286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:05.927896 15287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:05.927860 15289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:06:05.928880 15281 server_base.cc:1034] running on GCE node
I20260925 11:06:05.929603 15281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:06:05.931047 15281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:06:05.932257 15281 hybrid_clock.cc:648] HybridClock initialized: now 1790334365932194 us; error 33 us; skew 500 ppm
I20260925 11:06:05.932557 15281 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:06:05 dist-test-slave-trms krb5kdc[15120](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334365, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:06:05.935581 15281 init.cc:377] Logged in from keytab as kudu/127.24.159.252@KRBTEST.COM (short username kudu)
I20260925 11:06:05.935825 15274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } attempt: 4
I20260925 11:06:05.936913 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.936372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38156)
0925 11:06:05.936612 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.936710 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260925 11:06:05.937341 15281 webserver.cc:460] Webserver started at http://127.24.159.252:44047/ using document root <none> and password file <none>
I20260925 11:06:05.937829 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.937477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38160)
0925 11:06:05.937621 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.937706 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260925 11:06:05.938067 15274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:38651: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
I20260925 11:06:05.938256 15281 fs_manager.cc:359] Metadata directory not provided
I20260925 11:06:05.938349 15281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:06:05.938619 15281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:06:05.941579 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/data/instance:
uuid: "b0ba1c78c7a84835bf52a80620bd791d"
format_stamp: "Formatted at 2026-09-25 11:06:05 on dist-test-slave-trms"
server_key: "adedddb0cb744a95551765f040a9e011"
server_key_iv: "06e1036d7b458c34afd1b3a428a6ddf6"
server_key_version: "encryptionkey@0"
I20260925 11:06:05.942404 15281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/wal/instance:
uuid: "b0ba1c78c7a84835bf52a80620bd791d"
format_stamp: "Formatted at 2026-09-25 11:06:05 on dist-test-slave-trms"
server_key: "adedddb0cb744a95551765f040a9e011"
server_key_iv: "06e1036d7b458c34afd1b3a428a6ddf6"
server_key_version: "encryptionkey@0"
I20260925 11:06:05.947572 15281 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:06:05.951036 15296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:06:05.952368 15281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260925 11:06:05.952519 15281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/wal
uuid: "b0ba1c78c7a84835bf52a80620bd791d"
format_stamp: "Formatted at 2026-09-25 11:06:05 on dist-test-slave-trms"
server_key: "adedddb0cb744a95551765f040a9e011"
server_key_iv: "06e1036d7b458c34afd1b3a428a6ddf6"
server_key_version: "encryptionkey@0"
I20260925 11:06:05.952701 15281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:06:05.972702 15281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:06:05.976938 15281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:06:05.977161 15281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:06:05.982481 15200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } attempt: 3
I20260925 11:06:05.983623 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.983063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38164)
0925 11:06:05.983293 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.983405 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 11:06:05.984719 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:05.984361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38176)
0925 11:06:05.984531 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:05.984609 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260925 11:06:05.984936 15200 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:38651: Network error: Client connection negotiation failed: client connection to 127.24.159.252:38651: connect: Connection refused (error 111)
I20260925 11:06:05.995059 15281 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.252:38651
I20260925 11:06:05.995085 15347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.252:38651 every 8 connection(s)
I20260925 11:06:05.996399 15281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
I20260925 11:06:06.000111 15348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:06:06.001353 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 15281
I20260925 11:06:06.001511 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790334022505657-25215-0/minicluster-data/master-2/wal/instance
I20260925 11:06:06.001924 25215 external_mini_cluster.cc:1428] Setting key 87c7f79ae15e60bf7f3d4fda6a83ca3b
I20260925 11:06:06.008980 15348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:06:06 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
Sep 25 11:06:06 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334365, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:06:06.024456 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.003772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38184 (local address 127.24.159.252:38651)
0925 11:06:06.004561 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.004574 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.004615 (+ 41us) server_negotiation.cc:407] Connection header received
0925 11:06:06.006122 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.006149 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.006702 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.007021 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.008757 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.010103 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.011688 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.012396 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.012557 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.016754 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.016762 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.016773 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.016813 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.020142 (+ 3329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.020788 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.020797 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.020805 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.021005 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.021351 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.021356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.021359 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.022044 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.022353 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.022680 (+ 327us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.023114 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":364,"threads_started":1}
I20260925 11:06:06.026584 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.011580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:39913 (local address 127.0.0.1:52690)
0925 11:06:06.012460 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:06.012502 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:06.012712 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:06.013337 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:06.013342 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:06.013972 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:06.014270 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.014280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.015323 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.015327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.016760 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.016773 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.017513 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.017519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.017719 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.018290 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:06.018312 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:06.020717 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:06.024173 (+ 3456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.024182 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.024188 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.024591 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.024995 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.024998 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.025001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.025126 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.025620 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:06.025625 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:06.025779 (+ 154us) client_negotiation.cc:769] Sending connection context
0925 11:06:06.025931 (+ 152us) client_negotiation.cc:240] Negotiation successful
0925 11:06:06.026150 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":484,"threads_started":1}
I20260925 11:06:06.026633 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.011644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52690 (local address 127.24.159.254:39913)
0925 11:06:06.011857 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.011863 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.012589 (+ 726us) server_negotiation.cc:407] Connection header received
0925 11:06:06.012844 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.012848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.012923 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.013011 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.014408 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.015177 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.016900 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.017341 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.017465 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.020897 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.020904 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.020908 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.020949 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.024037 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.024744 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.024750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.024752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.024878 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.025228 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.025231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.025234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.025491 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.025633 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.026041 (+ 408us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.026293 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260925 11:06:06.028563 15348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:06:06 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334365, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
Sep 25 11:06:06 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:06:06.042797 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.029401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43892 (local address 127.24.159.253:45507)
0925 11:06:06.029576 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.029582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.029613 (+ 31us) server_negotiation.cc:407] Connection header received
0925 11:06:06.029758 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.029764 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.029848 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.029953 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.030754 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.031618 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.033159 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.033767 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.033915 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.037208 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.037214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.037217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.037250 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.040092 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.040862 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.040867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.040871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.041000 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.041347 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.041352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.041356 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.041706 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.041853 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.042232 (+ 379us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.042439 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260925 11:06:06.043011 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.029329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:45507 (local address 127.0.0.1:43892)
0925 11:06:06.029535 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:06.029554 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:06.029649 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:06.029983 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:06.029987 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:06.030409 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:06.030590 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.030600 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.031767 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.031770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.033010 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.033025 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.033937 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.033943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.034128 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.034717 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:06.034740 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:06.037020 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:06.040268 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.040273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.040277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.040753 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.041104 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.041109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.041112 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.041248 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.041828 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:06.041833 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:06.041979 (+ 146us) client_negotiation.cc:769] Sending connection context
0925 11:06:06.042100 (+ 121us) client_negotiation.cc:240] Negotiation successful
0925 11:06:06.042422 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260925 11:06:06.044102 15348 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:06:06.047255 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.030213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52698 (local address 127.24.159.254:39913)
0925 11:06:06.030392 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.030398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.030415 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:06:06.030462 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.030467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.030534 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.030622 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.032251 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.033198 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.034757 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.035312 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.035427 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.040811 (+ 5384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.040818 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.040827 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.040871 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.044487 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.045249 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.045255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.045258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.045487 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.045937 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.045942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.045946 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.046315 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.046605 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.046848 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.047019 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Sep 25 11:06:06 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334365, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
Sep 25 11:06:06 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334364, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
I20260925 11:06:06.057672 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.044905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38190 (local address 127.24.159.252:38651)
0925 11:06:06.045070 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.045075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.045092 (+ 17us) server_negotiation.cc:407] Connection header received
0925 11:06:06.045183 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.045188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.045251 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.045336 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.046111 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.047070 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.048447 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.048939 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.049058 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.052147 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.052154 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.052158 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.052194 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.055134 (+ 2940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.055693 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.055698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.055702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.055838 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.056208 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.056213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.056216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.056529 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.056632 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.057081 (+ 449us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.057396 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":110,"server-negotiator.queue_time_us":60}
I20260925 11:06:06.057693 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.044737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38190)
0925 11:06:06.044982 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:06.045000 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:06.045092 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:06.045367 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:06.045372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:06.045806 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:06.045998 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.046007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.047202 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.047206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.048322 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.048333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.049108 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.049114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.049358 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.049934 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:06.049950 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:06.051971 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:06.055253 (+ 3282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.055256 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.055259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.055587 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.055933 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.055937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.055940 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.056106 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.056631 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:06.056636 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:06.056791 (+ 155us) client_negotiation.cc:769] Sending connection context
0925 11:06:06.056970 (+ 179us) client_negotiation.cc:240] Negotiation successful
0925 11:06:06.057265 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260925 11:06:06.063366 15348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d: Bootstrap starting.
I20260925 11:06:06.063654 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.048977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43908 (local address 127.24.159.253:45507)
0925 11:06:06.049136 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.049141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.049170 (+ 29us) server_negotiation.cc:407] Connection header received
0925 11:06:06.049221 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.049226 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.049295 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.049404 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.050685 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.051515 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.052939 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.053384 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.053483 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.058266 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.058271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.058274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.058305 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.061215 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.061795 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.061800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.061803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.061992 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.062383 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.062387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.062390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.062701 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.062948 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.063135 (+ 187us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.063346 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:06:06.066902 15348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d: Neither blocks nor log segments found. Creating new log.
I20260925 11:06:06.067974 15348 log.cc:824] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d: Log is configured to *not* fsync() on all Append() calls
I20260925 11:06:06.068122 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:06:06.070597 15348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d: No bootstrap required, opened a new log
I20260925 11:06:06.070634 15175 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "806c8074e9b7436b93abc3319c76f451"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52698
I20260925 11:06:06.078011 15348 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } }
I20260925 11:06:06.078418 15348 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ba1c78c7a84835bf52a80620bd791d, State: Initialized, Role: FOLLOWER
I20260925 11:06:06.078827 15348 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } }
I20260925 11:06:06.079608 15354 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.079926 15354 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:06.080833 15348 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [sys.catalog]: configured and running, proceeding with master startup.
W20260925 11:06:06.089574 15365 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:06:06.089706 15365 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:06:06.087572 15281 master_runner.cc:186] Error getting master registration for 127.24.159.254:39913: OK, instance_id { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" instance_seqno: 1790334365031948 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:06:06.095431 15281 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 11:06:06.162142 15200 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } attempt: 4
Sep 25 11:06:06 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334364, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
I20260925 11:06:06.178750 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.162884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38196)
0925 11:06:06.163077 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:06.163098 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:06.163227 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:06.163627 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:06.163632 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:06.164163 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:06.164389 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.164402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.165880 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.165893 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.167396 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.167415 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.168429 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.168435 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.168624 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.169233 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:06.169257 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:06.171872 (+ 2615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:06.175764 (+ 3892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.175770 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.175774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.176324 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.176763 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.176769 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.176772 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.176963 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.177591 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:06.177596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:06.177840 (+ 244us) client_negotiation.cc:769] Sending connection context
0925 11:06:06.177993 (+ 153us) client_negotiation.cc:240] Negotiation successful
0925 11:06:06.178313 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 11:06:06.178763 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.163046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.24.159.252:38651)
0925 11:06:06.163260 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.163267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.163319 (+ 52us) server_negotiation.cc:407] Connection header received
0925 11:06:06.163395 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.163401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.163484 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.163596 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.164558 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.165720 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.167553 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.168287 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.168421 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.172065 (+ 3644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.172072 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.172078 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.172118 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.175575 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.176460 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.176467 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.176471 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.176645 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.177076 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.177081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.177084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.177485 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.177639 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.178120 (+ 481us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.178392 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 11:06:06.183409 15200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451: Bootstrap starting.
I20260925 11:06:06.186553 15200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451: Neither blocks nor log segments found. Creating new log.
I20260925 11:06:06.187814 15200 log.cc:824] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451: Log is configured to *not* fsync() on all Append() calls
I20260925 11:06:06.190853 15200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451: No bootstrap required, opened a new log
I20260925 11:06:06.196086 15200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } }
I20260925 11:06:06.196388 15200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 806c8074e9b7436b93abc3319c76f451, State: Initialized, Role: FOLLOWER
I20260925 11:06:06.196878 15200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } }
I20260925 11:06:06.197911 15368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.198323 15368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:06.199126 15200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:06:06.199236 15175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 11:06:06.199450 15175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:06:06.202929 15175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } }
I20260925 11:06:06.204476 15175 leader_election.cc:288] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [CANDIDATE]: Term 1 election: Requested vote from peers d035093a7c68457abc39c1da5823afc1 (127.24.159.253:45507), b0ba1c78c7a84835bf52a80620bd791d (127.24.159.252:38651)
I20260925 11:06:06.207594 15323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "806c8074e9b7436b93abc3319c76f451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b0ba1c78c7a84835bf52a80620bd791d"
I20260925 11:06:06.208078 15323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:06:06.211586 15323 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 806c8074e9b7436b93abc3319c76f451 in term 1.
I20260925 11:06:06.212355 15149 leader_election.cc:302] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 806c8074e9b7436b93abc3319c76f451, b0ba1c78c7a84835bf52a80620bd791d; no voters:
I20260925 11:06:06.212843 15368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:06:06.224136 15368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [term 1 LEADER]: Becoming Leader. State: Replica: 806c8074e9b7436b93abc3319c76f451, State: Running, Role: LEADER
I20260925 11:06:06.225204 15368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } }
W20260925 11:06:06.229223 15302 tablet.cc:2369] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:06:06.237262 15274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } attempt: 5
I20260925 11:06:06.237911 15378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 806c8074e9b7436b93abc3319c76f451. Latest consensus state: current_term: 1 leader_uuid: "806c8074e9b7436b93abc3319c76f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.239710 15378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [sys.catalog]: This master's current role is: LEADER
Sep 25 11:06:06 dist-test-slave-trms krb5kdc[15120](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334365, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
I20260925 11:06:06.240446 15133 master_runner.cc:186] Error getting master registration for 127.24.159.253:45507: OK, instance_id { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" instance_seqno: 1790334365517509 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:06:06.259024 15133 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 11:06:06.259415 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.238751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38202 (local address 127.24.159.252:38651)
0925 11:06:06.239072 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.239077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.239096 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:06:06.239160 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.239166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.239244 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.239337 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.240503 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.241540 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.243447 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.244091 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.244213 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.249262 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.249269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.249274 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.249315 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.253889 (+ 4574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.255087 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.255118 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.255123 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.255392 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.256137 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.256142 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.256144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.256745 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.257045 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.257867 (+ 822us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.258253 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210}
I20260925 11:06:06.260286 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.237797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:38651 (local address 127.0.0.1:38202)
0925 11:06:06.238379 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:06.238400 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:06.238526 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:06.239530 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:06.239536 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:06.240117 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:06.240344 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.240356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.241673 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.241680 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.243256 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:06.243286 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.244241 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.244248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:06.244521 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.245267 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:06.245290 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:06.248978 (+ 3688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:06.254103 (+ 5125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.254128 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.254132 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.254890 (+ 758us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.255604 (+ 714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:06.255609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:06.255613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:06.255948 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:06.257212 (+ 1264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:06.257217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:06.257477 (+ 260us) client_negotiation.cc:769] Sending connection context
0925 11:06:06.258435 (+ 958us) client_negotiation.cc:240] Negotiation successful
0925 11:06:06.259461 (+ 1026us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451}
W20260925 11:06:06.262451 15385 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:06:06.262568 15385 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:06:06.262709 15387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:06:06.264806 15387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:06:06.266091 15274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1: Bootstrap starting.
I20260925 11:06:06.269975 15274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1: Neither blocks nor log segments found. Creating new log.
I20260925 11:06:06.271311 15274 log.cc:824] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1: Log is configured to *not* fsync() on all Append() calls
W20260925 11:06:06.273046 15154 tablet.cc:2369] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:06:06.292348 15274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1: No bootstrap required, opened a new log
I20260925 11:06:06.292480 15323 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [term 1 FOLLOWER]: Refusing update from remote peer 806c8074e9b7436b93abc3319c76f451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:06:06.293643 15378 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:06:06.297387 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.238156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43910 (local address 127.24.159.253:45507)
0925 11:06:06.238940 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.238946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.252076 (+ 13130us) server_negotiation.cc:407] Connection header received
0925 11:06:06.253163 (+ 1087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.253170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.253253 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.253323 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.270365 (+ 17042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.271715 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.276625 (+ 4910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.277384 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.277494 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.286943 (+ 9449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.286950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.286955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.286998 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.290599 (+ 3601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.293161 (+ 2562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.293167 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.293170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.293353 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.294579 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.294584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.294588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.294961 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.295208 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.296746 (+ 1538us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.297031 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662}
I20260925 11:06:06.309017 15274 raft_consensus.cc:348] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } }
I20260925 11:06:06.310616 15274 raft_consensus.cc:729] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d035093a7c68457abc39c1da5823afc1, State: Initialized, Role: FOLLOWER
I20260925 11:06:06.312435 15274 consensus_queue.cc:260] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } }
I20260925 11:06:06.322260 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.240840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52710 (local address 127.24.159.254:39913)
0925 11:06:06.241040 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.241047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.278748 (+ 37701us) server_negotiation.cc:407] Connection header received
0925 11:06:06.278933 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.278939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.279023 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.279122 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.284868 (+ 5746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.286002 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.298543 (+ 12541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.299217 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.299318 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.311305 (+ 11987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.311314 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.311318 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.311380 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.315506 (+ 4126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.318998 (+ 3492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.319004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.319008 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.319200 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.319994 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.319999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.320003 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.320422 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.320673 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.321568 (+ 895us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.321854 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260925 11:06:06.323714 15354 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 806c8074e9b7436b93abc3319c76f451. Latest consensus state: current_term: 1 leader_uuid: "806c8074e9b7436b93abc3319c76f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.324074 15354 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:06.327252 15395 sys_catalog.cc:455] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.327695 15395 sys_catalog.cc:458] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:06.329447 15274 sys_catalog.cc:564] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:06:06.329507 15249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "806c8074e9b7436b93abc3319c76f451" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d035093a7c68457abc39c1da5823afc1"
I20260925 11:06:06.329859 15248 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:06:06.333498 15248 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [term 1 FOLLOWER]: Refusing update from remote peer 806c8074e9b7436b93abc3319c76f451: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:06:06.335465 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.278934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.24.159.252:38651)
0925 11:06:06.279129 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.279135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.300901 (+ 21766us) server_negotiation.cc:407] Connection header received
0925 11:06:06.301578 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.301584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.301685 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.301756 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.306986 (+ 5230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.308017 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.319427 (+ 11410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.320101 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.320238 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.323654 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:06.323661 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:06.323665 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:06.323713 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:06.331154 (+ 7441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.332066 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.332072 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.332076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.332260 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:06.332770 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:06.332775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:06.332778 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:06.333166 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:06.333440 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.334769 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.335040 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 11:06:06.336736 15249 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 806c8074e9b7436b93abc3319c76f451 in term 1.
I20260925 11:06:06.339094 15378 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:06:06.339516 15390 mvcc.cc:205] Tried to move back new op lower bound from 7333209564319399936 to 7333209564068458496. Current Snapshot: MvccSnapshot[applied={T|T < 7333209564319399936}]
I20260925 11:06:06.352315 15354 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "806c8074e9b7436b93abc3319c76f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.352593 15354 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:06.358793 15387 catalog_manager.cc:1347] Generated new cluster ID: 7cbbbafb13e844e19bc806d9936808e2
I20260925 11:06:06.358899 15387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:06:06.362572 15397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "806c8074e9b7436b93abc3319c76f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.362977 15397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [sys.catalog]: This master's current role is: LEADER
I20260925 11:06:06.378633 15387 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:06:06.381299 15387 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:06:06.406824 15411 sys_catalog.cc:455] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "806c8074e9b7436b93abc3319c76f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.407323 15411 sys_catalog.cc:458] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:06.407780 15395 sys_catalog.cc:455] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 806c8074e9b7436b93abc3319c76f451. Latest consensus state: current_term: 1 leader_uuid: "806c8074e9b7436b93abc3319c76f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.408051 15395 sys_catalog.cc:458] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:06.406006 15387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451: Generated new TSK 0
I20260925 11:06:06.410550 15387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260925 11:06:06.413273 15397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "806c8074e9b7436b93abc3319c76f451" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "806c8074e9b7436b93abc3319c76f451" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 39913 } } peers { permanent_uuid: "d035093a7c68457abc39c1da5823afc1" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 45507 } } peers { permanent_uuid: "b0ba1c78c7a84835bf52a80620bd791d" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 38651 } } }
I20260925 11:06:06.413538 15397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 806c8074e9b7436b93abc3319c76f451 [sys.catalog]: This master's current role is: LEADER
I20260925 11:06:06.441144 15413 catalog_manager.cc:1259] Loaded cluster ID: 7cbbbafb13e844e19bc806d9936808e2
I20260925 11:06:06.441334 15413 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1: loading cluster ID for follower catalog manager: success
I20260925 11:06:06.444815 15413 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1: acquiring CA information for follower catalog manager: success
I20260925 11:06:06.452319 15413 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d035093a7c68457abc39c1da5823afc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 11:06:06.550437 15279 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.536423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.24.159.253:45507)
0925 11:06:06.536619 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.536626 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.547460 (+ 10834us) server_negotiation.cc:407] Connection header received
0925 11:06:06.548809 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.548816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.548924 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.549003 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.550186 (+ 1183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43924: BlockingRecv error: recv got EOF from 127.0.0.1:43924 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260925 11:06:06.553879 15203 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.542744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.24.159.254:39913)
0925 11:06:06.542947 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.542954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.551879 (+ 8925us) server_negotiation.cc:407] Connection header received
0925 11:06:06.552576 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.552582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.552679 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.552751 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.553632 (+ 881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52718: BlockingRecv error: recv got EOF from 127.0.0.1:52718 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260925 11:06:06.557165 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.548009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38208 (local address 127.24.159.252:38651)
0925 11:06:06.548173 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.548179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.554997 (+ 6818us) server_negotiation.cc:407] Connection header received
0925 11:06:06.555663 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.555671 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.555752 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.555819 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.556939 (+ 1120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38208: BlockingRecv error: recv got EOF from 127.0.0.1:38208 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260925 11:06:06.614231 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.606333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43932 (local address 127.24.159.253:45507)
0925 11:06:06.606542 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.606548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.606752 (+ 204us) server_negotiation.cc:407] Connection header received
0925 11:06:06.607023 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.607030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.607206 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.607351 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:06.607872 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.609001 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.610996 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.611635 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.611750 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.612521 (+ 771us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:06.613330 (+ 809us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:06.613463 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.613712 (+ 249us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.613816 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260925 11:06:06.660830 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.658274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38224 (local address 127.24.159.252:38651)
0925 11:06:06.658473 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.658480 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.659000 (+ 520us) server_negotiation.cc:407] Connection header received
0925 11:06:06.659332 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.659339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.659482 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.659554 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.660601 (+ 1047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38224: BlockingRecv error: recv got EOF from 127.0.0.1:38224 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 11:06:06.688238 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.665629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43940 (local address 127.24.159.253:45507)
0925 11:06:06.665790 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.665795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.681148 (+ 15353us) server_negotiation.cc:407] Connection header received
0925 11:06:06.681708 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.681714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.681843 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.681921 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:06.682577 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.683763 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.685832 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.686608 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.686735 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.687367 (+ 632us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:06.687521 (+ 154us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:06.687579 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.687822 (+ 243us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.687913 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260925 11:06:06.698228 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.696085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.24.159.252:38651)
0925 11:06:06.696274 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.696280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.696756 (+ 476us) server_negotiation.cc:407] Connection header received
0925 11:06:06.696941 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.696946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.697041 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.697152 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.697984 (+ 832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38238: BlockingRecv error: recv got EOF from 127.0.0.1:38238 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260925 11:06:06.715093 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.708106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43954 (local address 127.24.159.253:45507)
0925 11:06:06.708288 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.708295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.708796 (+ 501us) server_negotiation.cc:407] Connection header received
0925 11:06:06.708994 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.709000 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.709091 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.709217 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:06.709670 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.710943 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.712940 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.713624 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.713769 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.714252 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:06.714422 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:06.714527 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.714652 (+ 125us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.714753 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260925 11:06:06.724115 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.722101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.24.159.252:38651)
0925 11:06:06.722295 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.722301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.722631 (+ 330us) server_negotiation.cc:407] Connection header received
0925 11:06:06.722839 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.722844 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.722930 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.723026 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.723851 (+ 825us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38242: BlockingRecv error: recv got EOF from 127.0.0.1:38242 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260925 11:06:06.740923 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.732408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.24.159.253:45507)
0925 11:06:06.732629 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.732635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.734754 (+ 2119us) server_negotiation.cc:407] Connection header received
0925 11:06:06.734972 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.734978 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.735065 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.735158 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:06.735630 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.736786 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.738854 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.739604 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.739762 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.740191 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:06.740329 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:06.740421 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.740557 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.740643 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
W20260925 11:06:06.752254 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.747953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.24.159.252:38651)
0925 11:06:06.748220 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.748226 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.750802 (+ 2576us) server_negotiation.cc:407] Connection header received
0925 11:06:06.750990 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.750996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.751102 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.751199 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.752047 (+ 848us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38256: BlockingRecv error: recv got EOF from 127.0.0.1:38256 (error 108)
Metrics: {"server-negotiator.queue_time_us":148}
I20260925 11:06:06.770473 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.763484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43966 (local address 127.24.159.253:45507)
0925 11:06:06.763692 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.763698 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.764305 (+ 607us) server_negotiation.cc:407] Connection header received
0925 11:06:06.764505 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.764511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.764598 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.764695 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:06.765226 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.766390 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.768248 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.768954 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.769095 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.769591 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:06.769762 (+ 171us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:06.769859 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.770009 (+ 150us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.770100 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260925 11:06:06.785437 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.783167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38262 (local address 127.24.159.252:38651)
0925 11:06:06.783359 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.783365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.783620 (+ 255us) server_negotiation.cc:407] Connection header received
0925 11:06:06.783970 (+ 350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.783977 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.784071 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.784145 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.785157 (+ 1012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38262: BlockingRecv error: recv got EOF from 127.0.0.1:38262 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260925 11:06:06.815853 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.808611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43978 (local address 127.24.159.253:45507)
0925 11:06:06.808839 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.808850 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.809446 (+ 596us) server_negotiation.cc:407] Connection header received
0925 11:06:06.809699 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.809705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.809813 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.809914 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:06.810428 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.811656 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.813640 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.814387 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.814524 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.815059 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:06.815212 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:06.815306 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.815472 (+ 166us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.815572 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260925 11:06:06.828109 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.822978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38270 (local address 127.24.159.252:38651)
0925 11:06:06.823178 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.823184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.826687 (+ 3503us) server_negotiation.cc:407] Connection header received
0925 11:06:06.826852 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.826859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.826961 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.827076 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.827888 (+ 812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38270: BlockingRecv error: recv got EOF from 127.0.0.1:38270 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 11:06:06.870539 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.864167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43984 (local address 127.24.159.253:45507)
0925 11:06:06.864375 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.864382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.864929 (+ 547us) server_negotiation.cc:407] Connection header received
0925 11:06:06.865122 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.865126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.865237 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.865313 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:06.865840 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.866669 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.868617 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.869128 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.869263 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.869776 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:06.869884 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:06.869955 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.870128 (+ 173us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.870225 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260925 11:06:06.877972 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.875950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.24.159.252:38651)
0925 11:06:06.876136 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.876143 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.876616 (+ 473us) server_negotiation.cc:407] Connection header received
0925 11:06:06.876777 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.876781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.876850 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.876927 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.877759 (+ 832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38272: BlockingRecv error: recv got EOF from 127.0.0.1:38272 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 11:06:06.951071 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:06.945976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43992 (local address 127.24.159.253:45507)
0925 11:06:06.946225 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.946232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.946498 (+ 266us) server_negotiation.cc:407] Connection header received
0925 11:06:06.946695 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.946699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.946769 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.946849 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:06.947258 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.948030 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.949449 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:06.949836 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:06.949947 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:06.950384 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:06.950502 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:06.950579 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:06.950724 (+ 145us) server_negotiation.cc:299] Negotiation successful
0925 11:06:06.950809 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260925 11:06:06.958166 15351 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:06.956389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38276 (local address 127.24.159.252:38651)
0925 11:06:06.956593 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:06.956599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:06.956945 (+ 346us) server_negotiation.cc:407] Connection header received
0925 11:06:06.957084 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:06.957088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:06.957159 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:06.957237 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:06.957970 (+ 733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38276: BlockingRecv error: recv got EOF from 127.0.0.1:38276 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260925 11:06:07.091938 15365 catalog_manager.cc:1259] Loaded cluster ID: 7cbbbafb13e844e19bc806d9936808e2
I20260925 11:06:07.092058 15365 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d: loading cluster ID for follower catalog manager: success
I20260925 11:06:07.094481 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:07.089236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44006 (local address 127.24.159.253:45507)
0925 11:06:07.089487 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:07.089494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:07.089672 (+ 178us) server_negotiation.cc:407] Connection header received
0925 11:06:07.089834 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:07.089838 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:07.089909 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:07.089986 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:07.090502 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.091161 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.092912 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.093284 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.093382 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:07.093815 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:07.093911 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:07.093968 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:07.094126 (+ 158us) server_negotiation.cc:299] Negotiation successful
0925 11:06:07.094201 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260925 11:06:07.095237 15365 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d: acquiring CA information for follower catalog manager: success
I20260925 11:06:07.097564 15365 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b0ba1c78c7a84835bf52a80620bd791d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 11:06:07.105530 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:07.100225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.24.159.252:38651)
0925 11:06:07.100442 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:07.100448 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:07.100705 (+ 257us) server_negotiation.cc:407] Connection header received
0925 11:06:07.100873 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:07.100878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:07.101001 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:07.101097 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:07.101467 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.102177 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.103504 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.103907 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.104011 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:07.104464 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:07.104956 (+ 492us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:07.105087 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:07.105182 (+ 95us) server_negotiation.cc:299] Negotiation successful
0925 11:06:07.105262 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260925 11:06:07.119043 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:07.111488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44008 (local address 127.24.159.253:45507)
0925 11:06:07.111701 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:07.111707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:07.113112 (+ 1405us) server_negotiation.cc:407] Connection header received
0925 11:06:07.113543 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:07.113548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:07.113631 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:07.113715 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:07.114433 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.115305 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.117153 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.117628 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.117748 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:07.118104 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:07.118234 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:07.118307 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:07.118710 (+ 403us) server_negotiation.cc:299] Negotiation successful
0925 11:06:07.118788 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260925 11:06:07.119043 15522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:07.111336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.24.159.254:39913)
0925 11:06:07.112347 (+ 1011us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:07.112351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:07.112374 (+ 23us) server_negotiation.cc:407] Connection header received
0925 11:06:07.112465 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:07.112469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:07.112607 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:07.112767 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:07.113322 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.114371 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.116297 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.116792 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.116901 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:07.117941 (+ 1040us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:07.118423 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:07.118509 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:07.118652 (+ 143us) server_negotiation.cc:299] Negotiation successful
0925 11:06:07.118732 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":913,"thread_start_us":815,"threads_started":1}
I20260925 11:06:07.119910 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:07.113081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38290 (local address 127.24.159.252:38651)
0925 11:06:07.113278 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:07.113284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:07.113941 (+ 657us) server_negotiation.cc:407] Connection header received
0925 11:06:07.114095 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:07.114098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:07.114158 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:07.114265 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:07.114932 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.115958 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.117702 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:07.118322 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:07.118442 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:07.119079 (+ 637us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:07.119204 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:07.119279 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:07.119582 (+ 303us) server_negotiation.cc:299] Negotiation successful
0925 11:06:07.119660 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260925 11:06:07.124002 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 15133
I20260925 11:06:07.124907 15195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:06:07.232908 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 15133
I20260925 11:06:07.249943 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 15207
I20260925 11:06:07.250986 15269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:06:07.356002 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 15207
I20260925 11:06:07.372005 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 15281
I20260925 11:06:07.373080 15343 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:06:07.477648 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 15281
2026-09-25T11:06:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4120 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 25 11:06:07 dist-test-slave-trms krb5kdc[15529](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 25 11:06:07 dist-test-slave-trms krb5kdc[15529](info): set up 2 sockets
Sep 25 11:06:07 dist-test-slave-trms krb5kdc[15529](info): commencing operation
krb5kdc: starting...
W20260925 11:06:08.502718 25215 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.011s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/test-user.keytab.
Sep 25 11:06:08 dist-test-slave-trms krb5kdc[15529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-25T11:06:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-25T11:06:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.159.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
Entry for principal kudu/127.24.159.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab.
I20260925 11:06:08.742651 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45197
--webserver_interface=127.24.159.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:45197,127.24.159.253:41745,127.24.159.252:43685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260925 11:06:09.053144 15542 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:06:09.053463 15542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:06:09.053561 15542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:06:09.053664 15542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:06:09.060791 15542 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:06:09.060879 15542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:06:09.060945 15542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:06:09.061013 15542 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:06:09.061092 15542 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:06:09.070703 15542 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:45197,127.24.159.253:41745,127.24.159.252:43685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.254.keytab
--principal=kudu/127.24.159.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.254:45197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.159.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:06:09.071913 15542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:06:09.073155 15542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:06:09.083837 15547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:09.083879 15548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:09.083900 15550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:06:09.084854 15542 server_base.cc:1034] running on GCE node
I20260925 11:06:09.085592 15542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:06:09.086915 15542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:06:09.088110 15542 hybrid_clock.cc:648] HybridClock initialized: now 1790334369088043 us; error 41 us; skew 500 ppm
I20260925 11:06:09.088416 15542 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:06:09 dist-test-slave-trms krb5kdc[15529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:06:09.091385 15542 init.cc:377] Logged in from keytab as kudu/127.24.159.254@KRBTEST.COM (short username kudu)
I20260925 11:06:09.092684 15542 webserver.cc:460] Webserver started at http://127.24.159.254:44239/ using document root <none> and password file <none>
I20260925 11:06:09.093291 15542 fs_manager.cc:359] Metadata directory not provided
I20260925 11:06:09.093374 15542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:06:09.093633 15542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:06:09.096418 15542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/data/instance:
uuid: "f8209b9c60f94f92ae36e54e9d32d95e"
format_stamp: "Formatted at 2026-09-25 11:06:09 on dist-test-slave-trms"
server_key: "97a210b4c084d0413a9392a7af6e93fd"
server_key_iv: "75f33ff382b3a99b4e706be2eb4fc46e"
server_key_version: "encryptionkey@0"
I20260925 11:06:09.097118 15542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/wal/instance:
uuid: "f8209b9c60f94f92ae36e54e9d32d95e"
format_stamp: "Formatted at 2026-09-25 11:06:09 on dist-test-slave-trms"
server_key: "97a210b4c084d0413a9392a7af6e93fd"
server_key_iv: "75f33ff382b3a99b4e706be2eb4fc46e"
server_key_version: "encryptionkey@0"
I20260925 11:06:09.101735 15542 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260925 11:06:09.104938 15557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:06:09.106164 15542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260925 11:06:09.106345 15542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/wal
uuid: "f8209b9c60f94f92ae36e54e9d32d95e"
format_stamp: "Formatted at 2026-09-25 11:06:09 on dist-test-slave-trms"
server_key: "97a210b4c084d0413a9392a7af6e93fd"
server_key_iv: "75f33ff382b3a99b4e706be2eb4fc46e"
server_key_version: "encryptionkey@0"
I20260925 11:06:09.106547 15542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:06:09.121546 15542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:06:09.125592 15542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:06:09.125829 15542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:06:09.142524 15542 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.254:45197
I20260925 11:06:09.142542 15608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.254:45197 every 8 connection(s)
I20260925 11:06:09.143880 15542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/data/info.pb
I20260925 11:06:09.147315 15609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:06:09.153767 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 15542
I20260925 11:06:09.153934 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-0/wal/instance
I20260925 11:06:09.153461 15609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:06:09.154414 25215 external_mini_cluster.cc:1428] Setting key bd883a9eeaaefa6b10b9b88d8544b9d7
Sep 25 11:06:09 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:06:09.175158 15612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.156565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53876 (local address 127.24.159.254:45197)
0925 11:06:09.158282 (+ 1717us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:09.158288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:09.158334 (+ 46us) server_negotiation.cc:407] Connection header received
0925 11:06:09.160054 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:09.160059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:09.160284 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:09.160432 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:09.161859 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.163052 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.164551 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.165120 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.165245 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:09.168887 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:09.168895 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:09.168906 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:09.168933 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:09.171431 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:09.172166 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:09.172173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:09.172175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:09.172304 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:09.172685 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:09.172688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:09.172690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:09.173123 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:09.173264 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:09.173836 (+ 572us) server_negotiation.cc:299] Negotiation successful
0925 11:06:09.174112 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":1509,"spinlock_wait_cycles":23808,"thread_start_us":1290,"threads_started":1}
I20260925 11:06:09.175189 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.156565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:45197 (local address 127.0.0.1:53876)
0925 11:06:09.157384 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.157449 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:09.159719 (+ 2270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:09.160524 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:09.160533 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:09.161224 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:09.161662 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:09.161685 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.163197 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.163203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:09.164414 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:09.164425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.165352 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.165358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:09.165534 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:09.166145 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:09.166172 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:09.168681 (+ 2509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:09.171550 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:09.171561 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:09.171572 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:09.172058 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:09.172411 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:09.172416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:09.172419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:09.172598 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:09.173252 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:09.173259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:09.173494 (+ 235us) client_negotiation.cc:769] Sending connection context
0925 11:06:09.173733 (+ 239us) client_negotiation.cc:240] Negotiation successful
0925 11:06:09.174309 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"mutex_wait_us":94,"thread_start_us":326,"threads_started":1}
I20260925 11:06:09.178287 15609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.159.253@KRBTEST.COM; defaulting to no policy
I20260925 11:06:09.179342 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.179003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41660)
0925 11:06:09.179149 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.179243 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 11:06:09.179630 15560 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
I20260925 11:06:09.180999 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.180695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41672)
0925 11:06:09.180814 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.180880 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260925 11:06:09.181169 15609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:41745: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab.
Entry for principal kudu/127.24.159.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab.
I20260925 11:06:09.208619 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.253:41745
--webserver_interface=127.24.159.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab
--principal=kudu/127.24.159.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:45197,127.24.159.253:41745,127.24.159.252:43685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:06:09.230473 15609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } attempt: 1
I20260925 11:06:09.231578 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.231139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41684)
0925 11:06:09.231315 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.231453 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260925 11:06:09.232491 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.232162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41694)
0925 11:06:09.232299 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.232363 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 11:06:09.232664 15609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:41745: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
I20260925 11:06:09.300870 15609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } attempt: 2
I20260925 11:06:09.301908 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.301453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41702)
0925 11:06:09.301678 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.301767 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 11:06:09.302942 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.302596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41716)
0925 11:06:09.302747 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.302811 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 11:06:09.303129 15609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:41745: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
I20260925 11:06:09.394344 15609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } attempt: 3
I20260925 11:06:09.395479 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.395065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41720)
0925 11:06:09.395248 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.395337 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 11:06:09.396409 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.396070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41722)
0925 11:06:09.396209 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.396275 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260925 11:06:09.396584 15609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:41745: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
W20260925 11:06:09.522394 15616 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:06:09.522729 15616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:06:09.522821 15616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:06:09.522945 15616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:06:09.530175 15616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:06:09.530288 15616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:06:09.530375 15616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:06:09.530437 15616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:06:09.530519 15616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:06:09.539786 15609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } attempt: 4
I20260925 11:06:09.540274 15616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:45197,127.24.159.253:41745,127.24.159.252:43685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.253.keytab
--principal=kudu/127.24.159.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.253:41745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.159.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:06:09.540934 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.540434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41736)
0925 11:06:09.540720 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.540802 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260925 11:06:09.541765 15616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:06:09.541839 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.541537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41746)
0925 11:06:09.541647 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.541724 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260925 11:06:09.542034 15609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.253:41745: Network error: Client connection negotiation failed: client connection to 127.24.159.253:41745: connect: Connection refused (error 111)
I20260925 11:06:09.543092 15616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260925 11:06:09.553730 15622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:09.553671 15621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:09.553838 15624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:06:09.554325 15616 server_base.cc:1034] running on GCE node
I20260925 11:06:09.555022 15616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:06:09.556267 15616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:06:09.557461 15616 hybrid_clock.cc:648] HybridClock initialized: now 1790334369557402 us; error 36 us; skew 500 ppm
I20260925 11:06:09.557745 15616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:06:09 dist-test-slave-trms krb5kdc[15529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:06:09.560523 15616 init.cc:377] Logged in from keytab as kudu/127.24.159.253@KRBTEST.COM (short username kudu)
I20260925 11:06:09.561779 15616 webserver.cc:460] Webserver started at http://127.24.159.253:44367/ using document root <none> and password file <none>
I20260925 11:06:09.562456 15616 fs_manager.cc:359] Metadata directory not provided
I20260925 11:06:09.562547 15616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:06:09.562829 15616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:06:09.565730 15616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/data/instance:
uuid: "d915309075694feb8f342c4fedf9ed85"
format_stamp: "Formatted at 2026-09-25 11:06:09 on dist-test-slave-trms"
server_key: "9d422013c812d64436593fe7d56816b0"
server_key_iv: "e4c58b278f0d77de7c7df03d85f5e6d4"
server_key_version: "encryptionkey@0"
I20260925 11:06:09.566699 15616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/wal/instance:
uuid: "d915309075694feb8f342c4fedf9ed85"
format_stamp: "Formatted at 2026-09-25 11:06:09 on dist-test-slave-trms"
server_key: "9d422013c812d64436593fe7d56816b0"
server_key_iv: "e4c58b278f0d77de7c7df03d85f5e6d4"
server_key_version: "encryptionkey@0"
I20260925 11:06:09.571633 15616 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260925 11:06:09.574759 15631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:06:09.575991 15616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260925 11:06:09.576128 15616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/wal
uuid: "d915309075694feb8f342c4fedf9ed85"
format_stamp: "Formatted at 2026-09-25 11:06:09 on dist-test-slave-trms"
server_key: "9d422013c812d64436593fe7d56816b0"
server_key_iv: "e4c58b278f0d77de7c7df03d85f5e6d4"
server_key_version: "encryptionkey@0"
I20260925 11:06:09.576313 15616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:06:09.599026 15616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:06:09.602854 15616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:06:09.603086 15616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:06:09.618914 15616 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.253:41745
I20260925 11:06:09.618933 15682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.253:41745 every 8 connection(s)
I20260925 11:06:09.620230 15616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/data/info.pb
I20260925 11:06:09.623889 15683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:06:09.630175 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 15616
I20260925 11:06:09.630381 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-1/wal/instance
I20260925 11:06:09.630764 25215 external_mini_cluster.cc:1428] Setting key b7680a39e238fc6e1c7315cdff423c9a
I20260925 11:06:09.630081 15683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:06:09 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:06:09.649010 15612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.632670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.24.159.254:45197)
0925 11:06:09.632828 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:09.632833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:09.634182 (+ 1349us) server_negotiation.cc:407] Connection header received
0925 11:06:09.636027 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:09.636033 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:09.636115 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:09.636201 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:09.637663 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.638380 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.639878 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.640281 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.640406 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:09.643607 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:09.643612 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:09.643616 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:09.643648 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:09.646051 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:09.646752 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:09.646756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:09.646759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:09.646875 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:09.647266 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:09.647269 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:09.647272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:09.647553 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:09.647694 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:09.648451 (+ 757us) server_negotiation.cc:299] Negotiation successful
0925 11:06:09.648682 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260925 11:06:09.649760 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.633299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:45197 (local address 127.0.0.1:53880)
0925 11:06:09.634021 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.634063 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:09.635752 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:09.636436 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:09.636444 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:09.636954 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:09.637487 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:09.637504 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.638543 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.638547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:09.639731 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:09.639744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.640492 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.640499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:09.640721 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:09.641326 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:09.641349 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:09.643389 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:09.646185 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:09.646214 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:09.646231 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:09.646639 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:09.646977 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:09.646983 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:09.646987 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:09.647155 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:09.647673 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:09.647681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:09.648022 (+ 341us) client_negotiation.cc:769] Sending connection context
0925 11:06:09.648278 (+ 256us) client_negotiation.cc:240] Negotiation successful
0925 11:06:09.648679 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":289,"threads_started":1}
I20260925 11:06:09.651490 15683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.159.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.159.252@KRBTEST.COM" created.
Sep 25 11:06:09 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
I20260925 11:06:09.665257 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.652264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:41760)
0925 11:06:09.652435 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.652455 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:09.652536 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:09.653452 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:09.653456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:09.653833 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:09.653961 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:09.653969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.655433 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.655437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:09.656240 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:09.656250 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.656996 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.657002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:09.657199 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:09.657705 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:09.657720 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:09.659700 (+ 1980us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:09.662953 (+ 3253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:09.662959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:09.662963 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:09.663302 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:09.663639 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:09.663644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:09.663648 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:09.663770 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:09.664318 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:09.664323 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:09.664470 (+ 147us) client_negotiation.cc:769] Sending connection context
0925 11:06:09.664585 (+ 115us) client_negotiation.cc:240] Negotiation successful
0925 11:06:09.664844 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":2}
I20260925 11:06:09.665298 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.652469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41760 (local address 127.24.159.253:41745)
0925 11:06:09.653029 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:09.653035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:09.653064 (+ 29us) server_negotiation.cc:407] Connection header received
0925 11:06:09.653111 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:09.653118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:09.653311 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:09.653440 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:09.654112 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.655301 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.656361 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.656863 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.656984 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:09.659878 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:09.659889 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:09.659904 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:09.659935 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:09.662826 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:09.663427 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:09.663434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:09.663436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:09.663543 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:09.663861 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:09.663866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:09.663870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:09.664204 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:09.664379 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:09.664731 (+ 352us) server_negotiation.cc:299] Negotiation successful
0925 11:06:09.664938 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":429,"thread_start_us":298,"threads_started":1}
I20260925 11:06:09.667683 15683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:06:09.668716 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.668273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:55524)
0925 11:06:09.668453 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.668559 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 11:06:09.668983 15635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
I20260925 11:06:09.670425 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.670092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:55534)
0925 11:06:09.670232 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.670316 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260925 11:06:09.670657 15683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:43685: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.159.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab.
Entry for principal kudu/127.24.159.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab.
I20260925 11:06:09.683269 25215 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
/tmp/dist-test-taskw_hhNs/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.252:43685
--webserver_interface=127.24.159.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab
--principal=kudu/127.24.159.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.159.212:33007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.159.254:45197,127.24.159.253:41745,127.24.159.252:43685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260925 11:06:09.719990 15683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } attempt: 1
I20260925 11:06:09.721089 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.720654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44482)
0925 11:06:09.720844 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.720943 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 11:06:09.722168 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.721859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44494)
0925 11:06:09.721984 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.722064 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260925 11:06:09.722406 15683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:43685: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
I20260925 11:06:09.790599 15683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } attempt: 2
I20260925 11:06:09.791685 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.791259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44504)
0925 11:06:09.791447 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.791536 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260925 11:06:09.792600 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.792260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44510)
0925 11:06:09.792410 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.792479 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260925 11:06:09.792783 15683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:43685: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
I20260925 11:06:09.841271 15609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } attempt: 5
Sep 25 11:06:09 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
I20260925 11:06:09.854218 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.842009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:56126)
0925 11:06:09.842243 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.842265 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:09.842414 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:09.842734 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:09.842738 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:09.843118 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:09.843298 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:09.843307 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.844267 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.844271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:09.845114 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:09.845126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.845859 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.845864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:09.846074 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:09.846752 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:09.846770 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:09.849058 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:09.851886 (+ 2828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:09.851892 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:09.851905 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:09.852273 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:09.852620 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:09.852625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:09.852629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:09.852765 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:09.853268 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:09.853273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:09.853420 (+ 147us) client_negotiation.cc:769] Sending connection context
0925 11:06:09.853545 (+ 125us) client_negotiation.cc:240] Negotiation successful
0925 11:06:09.853830 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260925 11:06:09.854213 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.842076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56126 (local address 127.24.159.253:41745)
0925 11:06:09.842278 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:09.842283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:09.842344 (+ 61us) server_negotiation.cc:407] Connection header received
0925 11:06:09.842537 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:09.842541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:09.842603 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:09.842684 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:09.843443 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.844130 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.845264 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:09.845702 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:09.845815 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:09.849257 (+ 3442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:09.849262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:09.849266 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:09.849299 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:09.851760 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:09.852382 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:09.852387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:09.852391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:09.852515 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:09.852865 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:09.852869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:09.852871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:09.853153 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:09.853286 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:09.853669 (+ 383us) server_negotiation.cc:299] Negotiation successful
0925 11:06:09.853873 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:06:09.855532 15609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } has no permanent_uuid. Determining permanent_uuid...
I20260925 11:06:09.856452 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.856085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44522)
0925 11:06:09.856240 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.856312 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260925 11:06:09.857271 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.856975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44532)
0925 11:06:09.857115 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.857180 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 11:06:09.857457 15609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:43685: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
I20260925 11:06:09.883968 15683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } attempt: 3
I20260925 11:06:09.885002 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.884589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44534)
0925 11:06:09.884761 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.884849 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260925 11:06:09.885967 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.885575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44540)
0925 11:06:09.885779 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.885854 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260925 11:06:09.886158 15683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:43685: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
I20260925 11:06:09.908650 15609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } attempt: 1
I20260925 11:06:09.909709 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.909260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44554)
0925 11:06:09.909444 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.909540 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260925 11:06:09.910648 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.910320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44564)
0925 11:06:09.910457 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.910526 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260925 11:06:09.911056 15609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:43685: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
I20260925 11:06:09.979298 15609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } attempt: 2
I20260925 11:06:09.980337 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.979884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44568)
0925 11:06:09.980100 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.980183 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260925 11:06:09.981361 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:09.981075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44574)
0925 11:06:09.981196 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:09.981263 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260925 11:06:09.981563 15609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:43685: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
W20260925 11:06:09.998484 15690 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260925 11:06:09.998828 15690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260925 11:06:09.998922 15690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260925 11:06:09.999033 15690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260925 11:06:10.006361 15690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260925 11:06:10.006460 15690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260925 11:06:10.006556 15690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260925 11:06:10.006619 15690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260925 11:06:10.006703 15690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260925 11:06:10.016709 15690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.159.212:33007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.159.254:45197,127.24.159.253:41745,127.24.159.252:43685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/krb5kdc/kudu_127.24.159.252.keytab
--principal=kudu/127.24.159.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.159.252:43685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.159.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 25 Sep 2026 09:24:16 UTC on ccycloud.quasar-kczgym.root.comops.site
build id 4672077
ASAN enabled
I20260925 11:06:10.017978 15690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260925 11:06:10.019341 15690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260925 11:06:10.029361 15683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } attempt: 4
I20260925 11:06:10.030741 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.030023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44584)
0925 11:06:10.030360 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:10.030510 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260925 11:06:10.031606 15696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:06:10.031972 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.031532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44592)
0925 11:06:10.031750 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:10.031832 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260925 11:06:10.032305 15695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260925 11:06:10.032337 15683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:43685: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
W20260925 11:06:10.032694 15698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260925 11:06:10.034296 15690 server_base.cc:1034] running on GCE node
I20260925 11:06:10.035043 15690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260925 11:06:10.036468 15690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260925 11:06:10.037643 15690 hybrid_clock.cc:648] HybridClock initialized: now 1790334370037572 us; error 45 us; skew 500 ppm
I20260925 11:06:10.037967 15690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 25 11:06:10 dist-test-slave-trms krb5kdc[15529](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260925 11:06:10.044608 15690 init.cc:377] Logged in from keytab as kudu/127.24.159.252@KRBTEST.COM (short username kudu)
I20260925 11:06:10.046099 15690 webserver.cc:460] Webserver started at http://127.24.159.252:43611/ using document root <none> and password file <none>
I20260925 11:06:10.046887 15690 fs_manager.cc:359] Metadata directory not provided
I20260925 11:06:10.046984 15690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260925 11:06:10.047329 15690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260925 11:06:10.050467 15690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/data/instance:
uuid: "e8a611628f3d4d498107a194ce960a5e"
format_stamp: "Formatted at 2026-09-25 11:06:10 on dist-test-slave-trms"
server_key: "617f694165f381945a86021665956846"
server_key_iv: "6bb656e7ba62e5d939940fbe0ca802ce"
server_key_version: "encryptionkey@0"
I20260925 11:06:10.051296 15690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/wal/instance:
uuid: "e8a611628f3d4d498107a194ce960a5e"
format_stamp: "Formatted at 2026-09-25 11:06:10 on dist-test-slave-trms"
server_key: "617f694165f381945a86021665956846"
server_key_iv: "6bb656e7ba62e5d939940fbe0ca802ce"
server_key_version: "encryptionkey@0"
I20260925 11:06:10.056514 15690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260925 11:06:10.060043 15705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260925 11:06:10.061364 15690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260925 11:06:10.061543 15690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/data,/tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/wal
uuid: "e8a611628f3d4d498107a194ce960a5e"
format_stamp: "Formatted at 2026-09-25 11:06:10 on dist-test-slave-trms"
server_key: "617f694165f381945a86021665956846"
server_key_iv: "6bb656e7ba62e5d939940fbe0ca802ce"
server_key_version: "encryptionkey@0"
I20260925 11:06:10.061743 15690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260925 11:06:10.079713 15690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260925 11:06:10.084014 15690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260925 11:06:10.084270 15690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260925 11:06:10.087749 15609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } attempt: 3
I20260925 11:06:10.088762 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.088283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44608)
0925 11:06:10.088505 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:10.088589 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260925 11:06:10.089876 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.089381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44610)
0925 11:06:10.089637 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:10.089738 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
W20260925 11:06:10.090126 15609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.159.252:43685: Network error: Client connection negotiation failed: client connection to 127.24.159.252:43685: connect: Connection refused (error 111)
I20260925 11:06:10.101222 15690 rpc_server.cc:307] RPC server started. Bound to: 127.24.159.252:43685
I20260925 11:06:10.101241 15756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.159.252:43685 every 8 connection(s)
I20260925 11:06:10.102612 15690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/data/info.pb
I20260925 11:06:10.105186 25215 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu as pid 15690
I20260925 11:06:10.105389 25215 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskw_hhNs/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790334022505657-25215-0/minicluster-data/master-2/wal/instance
I20260925 11:06:10.105948 25215 external_mini_cluster.cc:1428] Setting key 4b55436b4fd9abbe70ac283c4fbf426c
I20260925 11:06:10.106894 15757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260925 11:06:10.116621 15757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:06:10 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
Sep 25 11:06:10 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
I20260925 11:06:10.128435 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.107636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44614 (local address 127.24.159.252:43685)
0925 11:06:10.108380 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.108395 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.108447 (+ 52us) server_negotiation.cc:407] Connection header received
0925 11:06:10.110389 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.110419 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.110979 (+ 560us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.111422 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.112647 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.114701 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.115876 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.116908 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.117081 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.121276 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.121287 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.121298 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.121336 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.124206 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.124973 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.124981 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.124988 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.125192 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.125546 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.125552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.125555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.126084 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.126390 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.126810 (+ 420us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.127188 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":307,"threads_started":1}
I20260925 11:06:10.133488 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.119325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.254:45197 (local address 127.0.0.1:44380)
0925 11:06:10.120094 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:10.120138 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:10.120356 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:10.120792 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:10.120799 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:10.121563 (+ 764us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:10.121862 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.121872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.122947 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.122951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.124045 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.124057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.124985 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.124991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.125172 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.125736 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:10.125753 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:10.128019 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:10.130955 (+ 2936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.130962 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.130967 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.131495 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.131871 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.131875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.131877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.131986 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.132482 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:10.132489 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:10.132652 (+ 163us) client_negotiation.cc:769] Sending connection context
0925 11:06:10.132807 (+ 155us) client_negotiation.cc:240] Negotiation successful
0925 11:06:10.133086 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":368,"threads_started":1}
I20260925 11:06:10.133512 15612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.119339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.24.159.254:45197)
0925 11:06:10.119553 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.119560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.120224 (+ 664us) server_negotiation.cc:407] Connection header received
0925 11:06:10.120522 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.120527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.120616 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.120701 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.122004 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.122794 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.124176 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.124786 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.124937 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.128209 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.128215 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.128218 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.128250 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.130840 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.131607 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.131612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.131619 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.131779 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.132090 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.132094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.132096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.132356 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.132506 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.132933 (+ 427us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.133129 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260925 11:06:10.134815 15757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:06:10 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.254@KRBTEST.COM
Sep 25 11:06:10 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
I20260925 11:06:10.146551 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.131162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44386 (local address 127.24.159.254:45197)
0925 11:06:10.131780 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.131785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.131806 (+ 21us) server_negotiation.cc:407] Connection header received
0925 11:06:10.131847 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.131852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.131924 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.132005 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.133102 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.134040 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.135347 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.135938 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.136042 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.140420 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.140427 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.140432 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.140472 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.143438 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.144214 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.144219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.144223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.144414 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.144969 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.144974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.144977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.145335 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.145592 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.145860 (+ 268us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.146087 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":268,"threads_started":1}
I20260925 11:06:10.149739 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.135541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.253:41745 (local address 127.0.0.1:56132)
0925 11:06:10.135875 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:10.135894 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:10.136007 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:10.136325 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:10.136328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:10.136746 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:10.136889 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.136897 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.138159 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.138162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.139041 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.139053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.139888 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.139893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.140034 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.140514 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:10.140534 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:10.142965 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:10.146488 (+ 3523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.146493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.146497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.146980 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.147894 (+ 914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.147900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.147904 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.148077 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.148788 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:10.148792 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:10.148965 (+ 173us) client_negotiation.cc:769] Sending connection context
0925 11:06:10.149109 (+ 144us) client_negotiation.cc:240] Negotiation successful
0925 11:06:10.149416 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260925 11:06:10.150317 15687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.135663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56132 (local address 127.24.159.253:41745)
0925 11:06:10.135826 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.135833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.135965 (+ 132us) server_negotiation.cc:407] Connection header received
0925 11:06:10.136115 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.136121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.136198 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.136292 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.137028 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.138040 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.139176 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.139740 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.139875 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.143166 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.143173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.143178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.143219 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.146345 (+ 3126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.147636 (+ 1291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.147642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.147645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.147784 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.148205 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.148210 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.148213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.148573 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.148722 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.149257 (+ 535us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.149491 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260925 11:06:10.151242 15757 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } has no permanent_uuid. Determining permanent_uuid...
Sep 25 11:06:10 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.159.253@KRBTEST.COM
Sep 25 11:06:10 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.252@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
I20260925 11:06:10.163810 15761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.151888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44626)
0925 11:06:10.152107 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:10.152123 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:10.152223 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:10.152533 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:10.152537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:10.152988 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:10.153158 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.153168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.154278 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.154283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.155483 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.155498 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.156167 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.156174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.156398 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.156880 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:10.156903 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:10.158860 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:10.161581 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.161584 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.161587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.161935 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.162294 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.162299 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.162303 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.162432 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.162886 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:10.162890 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:10.163031 (+ 141us) client_negotiation.cc:769] Sending connection context
0925 11:06:10.163150 (+ 119us) client_negotiation.cc:240] Negotiation successful
0925 11:06:10.163431 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260925 11:06:10.163901 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.151936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44626 (local address 127.24.159.252:43685)
0925 11:06:10.152084 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.152090 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.152172 (+ 82us) server_negotiation.cc:407] Connection header received
0925 11:06:10.152324 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.152329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.152405 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.152496 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.153291 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.154138 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.155613 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.156009 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.156095 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.159029 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.159034 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.159037 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.159063 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.161452 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.162043 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.162046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.162049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.162163 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.162520 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.162525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.162528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.162781 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.162907 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.163249 (+ 342us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.163593 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":125,"server-negotiator.queue_time_us":53}
I20260925 11:06:10.163887 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.148247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56142 (local address 127.24.159.253:41745)
0925 11:06:10.148909 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.148914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.148936 (+ 22us) server_negotiation.cc:407] Connection header received
0925 11:06:10.149023 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.149028 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.149095 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.149199 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.151241 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.152238 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.153714 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.154381 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.154471 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.158937 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.158945 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.158949 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.158989 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.161352 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.162017 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.162022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.162026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.162156 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.162531 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.162536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.162539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.162796 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.163033 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.163296 (+ 263us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.163521 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":338,"threads_started":1}
I20260925 11:06:10.168836 25215 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260925 11:06:10.169364 15757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e: Bootstrap starting.
I20260925 11:06:10.172945 15757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e: Neither blocks nor log segments found. Creating new log.
I20260925 11:06:10.173971 15757 log.cc:824] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e: Log is configured to *not* fsync() on all Append() calls
I20260925 11:06:10.171706 15584 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f8209b9c60f94f92ae36e54e9d32d95e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44386
I20260925 11:06:10.176882 15757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e: No bootstrap required, opened a new log
I20260925 11:06:10.187249 15757 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } }
I20260925 11:06:10.187661 15757 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a611628f3d4d498107a194ce960a5e, State: Initialized, Role: FOLLOWER
I20260925 11:06:10.188081 15757 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } }
I20260925 11:06:10.188956 15765 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.189335 15765 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:10.189900 15757 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [sys.catalog]: configured and running, proceeding with master startup.
W20260925 11:06:10.198428 15776 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:06:10.198550 15776 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:06:10.196892 15690 master_runner.cc:186] Error getting master registration for 127.24.159.254:45197: OK, instance_id { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" instance_seqno: 1790334369137589 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:06:10.207682 15690 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 11:06:10.267374 15609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } attempt: 4
Sep 25 11:06:10 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.254@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
I20260925 11:06:10.282955 15611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.268044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44640)
0925 11:06:10.268306 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:10.268330 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:10.268486 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:10.268824 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:10.268829 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:10.269404 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:10.269606 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.269618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.270639 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.270646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.272156 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.272173 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.272900 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.272907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.273115 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.273716 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:10.273741 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:10.276375 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:10.280376 (+ 4001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.280382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.280387 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.280773 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.281151 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.281156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.281160 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.281295 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.281979 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:10.281983 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:10.282147 (+ 164us) client_negotiation.cc:769] Sending connection context
0925 11:06:10.282297 (+ 150us) client_negotiation.cc:240] Negotiation successful
0925 11:06:10.282576 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260925 11:06:10.282961 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.268132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44640 (local address 127.24.159.252:43685)
0925 11:06:10.268316 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.268320 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.268392 (+ 72us) server_negotiation.cc:407] Connection header received
0925 11:06:10.268626 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.268629 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.268698 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.268772 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.269766 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.270523 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.272311 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.272769 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.272854 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.276586 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.276597 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.276602 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.276639 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.280221 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.280885 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.280890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.280894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.281036 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.281407 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.281413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.281416 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.281866 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.282024 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.282399 (+ 375us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.282600 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 11:06:10.288924 15609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e: Bootstrap starting.
I20260925 11:06:10.292593 15609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e: Neither blocks nor log segments found. Creating new log.
I20260925 11:06:10.293627 15609 log.cc:824] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e: Log is configured to *not* fsync() on all Append() calls
I20260925 11:06:10.296304 15609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e: No bootstrap required, opened a new log
I20260925 11:06:10.301262 15609 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } }
I20260925 11:06:10.301565 15609 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8209b9c60f94f92ae36e54e9d32d95e, State: Initialized, Role: FOLLOWER
I20260925 11:06:10.301944 15609 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } }
I20260925 11:06:10.302760 15779 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.303136 15779 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:10.303901 15609 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:06:10.304088 15584 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260925 11:06:10.304327 15584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:06:10.307976 15584 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } }
W20260925 11:06:10.314312 15790 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260925 11:06:10.314440 15790 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260925 11:06:10.314772 15584 leader_election.cc:288] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [CANDIDATE]: Term 1 election: Requested vote from peers d915309075694feb8f342c4fedf9ed85 (127.24.159.253:41745), e8a611628f3d4d498107a194ce960a5e (127.24.159.252:43685)
I20260925 11:06:10.316123 15732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e8a611628f3d4d498107a194ce960a5e"
I20260925 11:06:10.316612 15732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:06:10.320416 15732 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8209b9c60f94f92ae36e54e9d32d95e in term 1.
I20260925 11:06:10.321194 15561 leader_election.cc:302] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8a611628f3d4d498107a194ce960a5e, f8209b9c60f94f92ae36e54e9d32d95e; no voters:
I20260925 11:06:10.321679 15779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [term 1 FOLLOWER]: Leader election won for term 1
I20260925 11:06:10.313506 15542 master_runner.cc:186] Error getting master registration for 127.24.159.253:41745: OK, instance_id { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" instance_seqno: 1790334369614322 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260925 11:06:10.323453 15542 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260925 11:06:10.331528 15683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } attempt: 5
I20260925 11:06:10.332890 15779 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [term 1 LEADER]: Becoming Leader. State: Replica: f8209b9c60f94f92ae36e54e9d32d95e, State: Running, Role: LEADER
I20260925 11:06:10.333580 15779 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } }
W20260925 11:06:10.336382 15711 tablet.cc:2369] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 25 11:06:10 dist-test-slave-trms krb5kdc[15529](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790334369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.159.253@KRBTEST.COM for kudu/127.24.159.252@KRBTEST.COM
I20260925 11:06:10.342819 15794 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8209b9c60f94f92ae36e54e9d32d95e. Latest consensus state: current_term: 1 leader_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.343217 15794 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [sys.catalog]: This master's current role is: LEADER
I20260925 11:06:10.352748 15685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.332152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.159.252:43685 (local address 127.0.0.1:44648)
0925 11:06:10.332519 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0925 11:06:10.332538 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0925 11:06:10.332652 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0925 11:06:10.333510 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0925 11:06:10.333515 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0925 11:06:10.334017 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0925 11:06:10.334248 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.334260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.335848 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.335853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.337285 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0925 11:06:10.337302 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.338587 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.338594 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0925 11:06:10.338824 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.339482 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0925 11:06:10.339505 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0925 11:06:10.342279 (+ 2774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0925 11:06:10.346494 (+ 4215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.346500 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.346504 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.347670 (+ 1166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.348303 (+ 633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0925 11:06:10.348308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0925 11:06:10.348332 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0925 11:06:10.348669 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0925 11:06:10.349674 (+ 1005us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0925 11:06:10.349678 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0925 11:06:10.350233 (+ 555us) client_negotiation.cc:769] Sending connection context
0925 11:06:10.350541 (+ 308us) client_negotiation.cc:240] Negotiation successful
0925 11:06:10.351451 (+ 910us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260925 11:06:10.353195 15760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.332884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44648 (local address 127.24.159.252:43685)
0925 11:06:10.333060 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.333066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.333085 (+ 19us) server_negotiation.cc:407] Connection header received
0925 11:06:10.333147 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.333153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.333227 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.333317 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.334440 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.335613 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.337469 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.338180 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.338366 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.342549 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.342556 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.342565 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.342604 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.346302 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.347834 (+ 1532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.347840 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.347844 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.348156 (+ 312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.348816 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.348821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.348849 (+ 28us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.349486 (+ 637us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.350621 (+ 1135us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.351116 (+ 495us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.351807 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260925 11:06:10.358379 15798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260925 11:06:10.359941 15683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85: Bootstrap starting.
I20260925 11:06:10.360625 15798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260925 11:06:10.364815 15683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85: Neither blocks nor log segments found. Creating new log.
I20260925 11:06:10.367332 15683 log.cc:824] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85: Log is configured to *not* fsync() on all Append() calls
W20260925 11:06:10.378443 15563 tablet.cc:2369] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260925 11:06:10.382910 15763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.345688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56146 (local address 127.24.159.253:41745)
0925 11:06:10.345872 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.345878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.348391 (+ 2513us) server_negotiation.cc:407] Connection header received
0925 11:06:10.349306 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.349312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.349411 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.349482 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.353477 (+ 3995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.354473 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.362423 (+ 7950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.363147 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.363255 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.367088 (+ 3833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.367097 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.367102 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.367151 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.371005 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.380381 (+ 9376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.380387 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.380391 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.380581 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.381078 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.381083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.381087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.381469 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.381709 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.382294 (+ 585us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.382556 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260925 11:06:10.395524 15683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85: No bootstrap required, opened a new log
I20260925 11:06:10.400055 15732 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [term 1 FOLLOWER]: Refusing update from remote peer f8209b9c60f94f92ae36e54e9d32d95e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:06:10.400967 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.346760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.24.159.254:45197)
0925 11:06:10.346933 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.346938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.354021 (+ 7083us) server_negotiation.cc:407] Connection header received
0925 11:06:10.354260 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.354266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.354356 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.354449 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.377248 (+ 22799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.378415 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.381537 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.382293 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.382395 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.390783 (+ 8388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.390790 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.390795 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.390838 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.394467 (+ 3629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.396808 (+ 2341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.396813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.396817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.396981 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.398087 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.398092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.398096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.398504 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.398737 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.400381 (+ 1644us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.400646 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260925 11:06:10.401175 15794 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:06:10.409924 15802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:10.354742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44650 (local address 127.24.159.252:43685)
0925 11:06:10.356895 (+ 2153us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.356900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.375367 (+ 18467us) server_negotiation.cc:407] Connection header received
0925 11:06:10.375680 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.375686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.375774 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.375855 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.377937 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.379272 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.384924 (+ 5652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:10.385633 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:10.385739 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:10.390361 (+ 4622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0925 11:06:10.390369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0925 11:06:10.390374 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0925 11:06:10.390426 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0925 11:06:10.394633 (+ 4207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.404577 (+ 9944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.404582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.404586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.404834 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0925 11:06:10.406082 (+ 1248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0925 11:06:10.406087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0925 11:06:10.406091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0925 11:06:10.406794 (+ 703us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0925 11:06:10.407206 (+ 412us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:10.408767 (+ 1561us) server_negotiation.cc:299] Negotiation successful
0925 11:06:10.409218 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2022,"thread_start_us":3528,"threads_started":1}
I20260925 11:06:10.423931 15683 raft_consensus.cc:348] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } }
I20260925 11:06:10.424587 15683 raft_consensus.cc:729] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d915309075694feb8f342c4fedf9ed85, State: Initialized, Role: FOLLOWER
I20260925 11:06:10.424989 15765 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8209b9c60f94f92ae36e54e9d32d95e. Latest consensus state: current_term: 1 leader_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.425308 15765 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:10.425379 15683 consensus_queue.cc:260] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } }
I20260925 11:06:10.433203 15804 mvcc.cc:205] Tried to move back new op lower bound from 7333209581145001984 to 7333209580892401664. Current Snapshot: MvccSnapshot[applied={T|T < 7333209581145001984}]
I20260925 11:06:10.434495 15805 mvcc.cc:205] Tried to move back new op lower bound from 7333209581145001984 to 7333209580892401664. Current Snapshot: MvccSnapshot[applied={T|T < 7333209581145001984}]
I20260925 11:06:10.435613 15779 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.435923 15779 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [sys.catalog]: This master's current role is: LEADER
I20260925 11:06:10.436815 15765 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.437042 15765 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:10.438868 15807 sys_catalog.cc:455] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.439285 15807 sys_catalog.cc:458] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:10.440315 15683 sys_catalog.cc:564] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [sys.catalog]: configured and running, proceeding with master startup.
I20260925 11:06:10.440696 15657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [term 0 FOLLOWER]: Advancing to term 1
I20260925 11:06:10.441094 15658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d915309075694feb8f342c4fedf9ed85"
I20260925 11:06:10.445192 15657 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [term 1 FOLLOWER]: Refusing update from remote peer f8209b9c60f94f92ae36e54e9d32d95e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260925 11:06:10.446132 15794 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260925 11:06:10.447988 15798 catalog_manager.cc:1347] Generated new cluster ID: 6e318a72f7f24ac6bcb1cab577615a99
I20260925 11:06:10.448089 15798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260925 11:06:10.469365 15658 raft_consensus.cc:2396] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f8209b9c60f94f92ae36e54e9d32d95e for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260925 11:06:10.472348 15807 sys_catalog.cc:455] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8209b9c60f94f92ae36e54e9d32d95e. Latest consensus state: current_term: 1 leader_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.472718 15807 sys_catalog.cc:458] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:10.472982 15813 sys_catalog.cc:455] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.473340 15813 sys_catalog.cc:458] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85 [sys.catalog]: This master's current role is: FOLLOWER
I20260925 11:06:10.477312 15794 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8209b9c60f94f92ae36e54e9d32d95e" member_type: VOTER last_known_addr { host: "127.24.159.254" port: 45197 } } peers { permanent_uuid: "d915309075694feb8f342c4fedf9ed85" member_type: VOTER last_known_addr { host: "127.24.159.253" port: 41745 } } peers { permanent_uuid: "e8a611628f3d4d498107a194ce960a5e" member_type: VOTER last_known_addr { host: "127.24.159.252" port: 43685 } } }
I20260925 11:06:10.477591 15794 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e [sys.catalog]: This master's current role is: LEADER
I20260925 11:06:10.485565 15798 catalog_manager.cc:1370] Generated new certificate authority record
I20260925 11:06:10.487295 15798 catalog_manager.cc:1504] Loading token signing keys...
I20260925 11:06:10.513190 15824 catalog_manager.cc:1259] Loaded cluster ID: 6e318a72f7f24ac6bcb1cab577615a99
I20260925 11:06:10.513298 15824 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85: loading cluster ID for follower catalog manager: success
I20260925 11:06:10.517120 15824 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85: acquiring CA information for follower catalog manager: success
W20260925 11:06:10.520529 15824 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260925 11:06:10.526513 15798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8209b9c60f94f92ae36e54e9d32d95e: Generated new TSK 0
I20260925 11:06:10.528187 15798 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260925 11:06:10.677145 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.664912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.24.159.253:41745)
0925 11:06:10.665096 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.665105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.675042 (+ 9937us) server_negotiation.cc:407] Connection header received
0925 11:06:10.675566 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.675572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.675710 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.675789 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.676873 (+ 1084us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56148: BlockingRecv error: recv got EOF from 127.0.0.1:56148 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260925 11:06:10.677547 15762 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.665915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44406 (local address 127.24.159.254:45197)
0925 11:06:10.666077 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.666083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.675833 (+ 9750us) server_negotiation.cc:407] Connection header received
0925 11:06:10.676103 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.676109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.676190 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.676256 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.677303 (+ 1047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44406: BlockingRecv error: recv got EOF from 127.0.0.1:44406 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260925 11:06:10.680006 15802 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.678231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44652 (local address 127.24.159.252:43685)
0925 11:06:10.678420 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.678426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.678657 (+ 231us) server_negotiation.cc:407] Connection header received
0925 11:06:10.678899 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.678905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.679003 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.679100 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.679776 (+ 676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44652: BlockingRecv error: recv got EOF from 127.0.0.1:44652 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260925 11:06:10.724797 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.715022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56164 (local address 127.24.159.253:41745)
0925 11:06:10.715228 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.715235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.722515 (+ 7280us) server_negotiation.cc:407] Connection header received
0925 11:06:10.723052 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.723059 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.723390 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.723471 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.724554 (+ 1083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56164: BlockingRecv error: recv got EOF from 127.0.0.1:56164 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260925 11:06:10.734828 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.732279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.24.159.253:41745)
0925 11:06:10.732444 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.732449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.732532 (+ 83us) server_negotiation.cc:407] Connection header received
0925 11:06:10.732786 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.732792 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.732860 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.732944 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.734035 (+ 1091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56170: BlockingRecv error: recv got EOF from 127.0.0.1:56170 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260925 11:06:10.741011 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.739482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56180 (local address 127.24.159.253:41745)
0925 11:06:10.739664 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.739669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.739767 (+ 98us) server_negotiation.cc:407] Connection header received
0925 11:06:10.739994 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.740001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.740078 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.740156 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.740819 (+ 663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56180: BlockingRecv error: recv got EOF from 127.0.0.1:56180 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260925 11:06:10.750052 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.748338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56182 (local address 127.24.159.253:41745)
0925 11:06:10.748533 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.748540 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.748844 (+ 304us) server_negotiation.cc:407] Connection header received
0925 11:06:10.749032 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.749038 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.749109 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.749211 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.749859 (+ 648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56182: BlockingRecv error: recv got EOF from 127.0.0.1:56182 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260925 11:06:10.763163 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.761214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56194 (local address 127.24.159.253:41745)
0925 11:06:10.761384 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.761390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.761887 (+ 497us) server_negotiation.cc:407] Connection header received
0925 11:06:10.762052 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.762057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.762138 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.762277 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.762947 (+ 670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56194: BlockingRecv error: recv got EOF from 127.0.0.1:56194 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260925 11:06:10.793584 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.791613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56198 (local address 127.24.159.253:41745)
0925 11:06:10.791805 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.791811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.791897 (+ 86us) server_negotiation.cc:407] Connection header received
0925 11:06:10.792213 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.792219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.792336 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.792447 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.793287 (+ 840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56198: BlockingRecv error: recv got EOF from 127.0.0.1:56198 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260925 11:06:10.834421 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.832088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56212 (local address 127.24.159.253:41745)
0925 11:06:10.832296 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.832303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.832872 (+ 569us) server_negotiation.cc:407] Connection header received
0925 11:06:10.833093 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.833100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.833187 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.833296 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.834135 (+ 839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56212: BlockingRecv error: recv got EOF from 127.0.0.1:56212 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260925 11:06:10.907569 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:10.905108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.24.159.253:41745)
0925 11:06:10.905290 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:10.905298 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:10.905908 (+ 610us) server_negotiation.cc:407] Connection header received
0925 11:06:10.906107 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:10.906116 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:10.906297 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:10.906414 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:10.907332 (+ 918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56218: BlockingRecv error: recv got EOF from 127.0.0.1:56218 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260925 11:06:11.040921 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:11.038949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.24.159.253:41745)
0925 11:06:11.039177 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:11.039183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:11.039460 (+ 277us) server_negotiation.cc:407] Connection header received
0925 11:06:11.039674 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:11.039680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:11.039777 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:11.039883 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:11.040698 (+ 815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56234: BlockingRecv error: recv got EOF from 127.0.0.1:56234 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260925 11:06:11.200444 15776 catalog_manager.cc:1259] Loaded cluster ID: 6e318a72f7f24ac6bcb1cab577615a99
I20260925 11:06:11.200546 15776 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e: loading cluster ID for follower catalog manager: success
I20260925 11:06:11.203011 15776 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e: acquiring CA information for follower catalog manager: success
I20260925 11:06:11.204795 15776 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8a611628f3d4d498107a194ce960a5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260925 11:06:11.302258 15763 negotiation.cc:337] Failed RPC negotiation. Trace:
0925 11:06:11.300413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56242 (local address 127.24.159.253:41745)
0925 11:06:11.300649 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:11.300656 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:11.300877 (+ 221us) server_negotiation.cc:407] Connection header received
0925 11:06:11.301048 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:11.301052 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:11.301125 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:11.301242 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0925 11:06:11.301973 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56242: BlockingRecv error: recv got EOF from 127.0.0.1:56242 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260925 11:06:11.523264 15824 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d915309075694feb8f342c4fedf9ed85: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260925 11:06:11.824967 15889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:11.819732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56256 (local address 127.24.159.253:41745)
0925 11:06:11.820183 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:11.820188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:11.820264 (+ 76us) server_negotiation.cc:407] Connection header received
0925 11:06:11.820428 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:11.820432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:11.820592 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:11.820701 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:11.821082 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.821744 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.823015 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.823398 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.823484 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:11.823904 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:11.824408 (+ 504us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:11.824517 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:11.824593 (+ 76us) server_negotiation.cc:299] Negotiation successful
0925 11:06:11.824663 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":243,"threads_started":1}
I20260925 11:06:11.836114 15895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:11.830762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44654 (local address 127.24.159.252:43685)
0925 11:06:11.831180 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:11.831187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:11.831225 (+ 38us) server_negotiation.cc:407] Connection header received
0925 11:06:11.831450 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:11.831454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:11.831601 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:11.831744 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:11.832106 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.832774 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.834033 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.834487 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.834597 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:11.835029 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:11.835529 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:11.835664 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:11.835745 (+ 81us) server_negotiation.cc:299] Negotiation successful
0925 11:06:11.835825 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":210,"threads_started":1}
I20260925 11:06:11.849176 15895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:11.842914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.24.159.252:43685)
0925 11:06:11.843052 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:11.843056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:11.843572 (+ 516us) server_negotiation.cc:407] Connection header received
0925 11:06:11.843712 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:11.843717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:11.843796 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:11.843887 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:11.844441 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.845125 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.847270 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.847863 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.848022 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:11.848517 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:11.848619 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:11.848691 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:11.848853 (+ 162us) server_negotiation.cc:299] Negotiation successful
0925 11:06:11.848924 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260925 11:06:11.849151 15889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:11.841741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56270 (local address 127.24.159.253:41745)
0925 11:06:11.841924 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:11.841931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:11.842868 (+ 937us) server_negotiation.cc:407] Connection header received
0925 11:06:11.843286 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:11.843293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:11.843381 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:11.843443 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:11.844240 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.845298 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.847312 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.847749 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.847929 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:11.848393 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:11.848532 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:11.848612 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:11.848783 (+ 171us) server_negotiation.cc:299] Negotiation successful
0925 11:06:11.848866 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260925 11:06:11.850600 15901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0925 11:06:11.841739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44412 (local address 127.24.159.254:45197)
0925 11:06:11.842324 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0925 11:06:11.842330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0925 11:06:11.842704 (+ 374us) server_negotiation.cc:407] Connection header received
0925 11:06:11.843018 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0925 11:06:11.843024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0925 11:06:11.843254 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0925 11:06:11.843430 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0925 11:06:11.844180 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.845308 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.847728 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0925 11:06:11.848255 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0925 11:06:11.848359 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0925 11:06:11.849069 (+ 710us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0925 11:06:11.849850 (+ 781us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0925 11:06:11.849953 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0925 11:06:11.850089 (+ 136us) server_negotiation.cc:299] Negotiation successful
0925 11:06:11.850332 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":312,"threads_started":1}
I20260925 11:06:11.854853 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 15542
I20260925 11:06:11.855751 15604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:06:11.964263 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 15542
W20260925 11:06:11.981077 15708 connection.cc:535] server connection from 127.0.0.1:44640 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260925 11:06:11.981499 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 15616
I20260925 11:06:11.982533 15678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:06:12.094389 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 15616
I20260925 11:06:12.110350 25215 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu pid 15690
I20260925 11:06:12.111402 15752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260925 11:06:12.218135 25215 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskw_hhNs/build/asan/bin/kudu with pid 15690
2026-09-25T11:06:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4740 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (8861 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (349676 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST